M&A, Before and After: What Founders Need to Know

Martin Casado, Blake Kim, and Zoran Basich

Welcome to the a16z podcast. Today we’re talking about the mindsets and frameworks founders should know about when navigating the mergers and acquisitions or M&A process, both before and after – including how to think about the pricing dynamics, factors that go into the decision-making process, and what to expect from the integration once the deal is done.

A16z editorial partner Zoran Basich recently talked to two a16z experts here to give us their big-picture view of the most important things to know – for founders seeking to acquire companies and how they might think about it, or those considering selling a company, or just those deciding to merge with an acquirer.

Blake Kim is a partner on our Enterprise Network team and a former investment banker who works with companies on strategic partnerships; he also recently co-wrote a post on Future outlining all the different exit options and considerations for companies. And general partner Martin Casado discusses common M&A issues and shares his experiences both as observer and participant – including the challenges of integration, which he saw from the inside with Nicira, which he cofounded and was acquired by VMware for $1.26 billion in 2012, and where he remained for years to lead its networking and security business unit.

As a reminder, none of the following should be taken as investment advice. Please see a16z.com/disclosures for more important information.

They start the discussion by outlining the frameworks for understanding M&A dynamics, including the “kingmaking dynamic” and the difference between “selling your company” and “getting acquired.”

  • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

  • Blake Kim is Enterprise Lead in a16z's Capital Network Group, advising companies on capital raising, M&A, and public markets. He has worked at Bear Stearns, Lehman Brothers and Thomas Weisel Partners/Stifel.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Crypto at Congress: ‘Watershed’ Moment for Regulation and Web3

Tomicah Tillemann and Zoran Basich

Welcome to 16 Minutes, our podcast where we discuss tech trends in the news and their impact on the long arc of innovation. Today’s topic is crypto regulation, and specifically, two recent federal government hearings in the news that were focused on crypto and therefore the related trend of web3. In contrast to the model of web2 — typified by very broadly used but also very centralized platforms run by corporations — web3 refers to the idea of a new internet enabled by crypto that is owned by builders and users.

The first hearing that took place was at the House Committee on Financial Services, featuring six crypto company CEOs and resulting in a five-hour session that prompted headlines like “Congress Gets a Crash Course on Cryptocurrency.”

Then, the U.S. Senate’s Banking, Housing, and Urban Affairs Committee held its own hearing, this time focused on stablecoins, which are privately issued cryptocurrencies that are pegged to a stable asset such as the U.S. dollar, and are used in decentralized financial services.

We’ve covered crypto regulatory issues on 16 Minutes before with a16z experts, including an episode with former federal prosecutor  Katie Haun and former New York Stock Exchange regulatory chief Anthony Albanese. That discussion, which you can find in this feed under episode #50, was about a proposal by the Treasury Department’s financial crimes enforcement arm that included provisions for digital asset reporting.

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.)

All of these hearings are also connected to the broader question of innovation, and keeping the U.S. competitive on a global stage.

So with that context, our guest today is a16z global head of policy Tomicah Tillemann, who before joining a16z served as senior advisor to two secretaries of state. He reports on the hearings and their significance, and gives a quick pulse-check on where we are with crypto regulation right now.

  • Tomicah Tillemann is global head of policy at a16z. Previously he served as a senior advisor to now-President Joseph Biden and two secretaries of state.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

How ‘Hyperscalers’ are Innovating — and Competing — in the Data Center

Nick McKeown, Martin Casado, and Zoran Basich

Innovation in the data center has been constrained by the traditional model of suppliers providing fixed-function chips that limit how much the biggest data center operators can differentiate. But programmable chips have emerged that allow these companies to not only increase performance, but innovate throughout the pipeline, from operating system to networking interface to user application.

This is a major trend among “hyperscalers,” which are some of the world’s most well known companies running massive data centers with tens of thousands of servers. We’re talking about companies like Amazon, Facebook, Microsoft, Google, Apple, Alibaba, Tencent.

To talk about the trends in data centers and how software may be “eating the world of the data center,” we talked this summer to two experts. Martin Casado is an a16z general partner focused on enterprise investing. Before that he was a pioneer in the software-defined networking movement and the cofounder of Nicira, which was acquired by VMWare. (Martin has written frequently on infrastructure and data-center issues and has appeared on many a16z podcasts on these topics.)

He’s joined by Nick McKeown, a Stanford professor of computer science who has founded multiple companies (and was Martin’s cofounder at Nicira) and has worked with hyperscalers to innovate within their data centers. After this podcast was recorded, Nick was appointed Senior Vice President and General Manager of a new Intel organization, the Network and Edge Group. The podcast begins with Nick, talking about the sheer scale of data-center traffic. 

  • Nick McKeown

  • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Kickstarting Network Effects

Paul Davison, Alexis Ohanian, Andrew Chen, and Das Rush

Network effects can be found powering almost every major technology company, from messaging apps and workplace collaboration tools, like Slack and Zoom, to marketplaces, like Airbnb and Instacart to even the internet itself. In this podcast, we look at the role of network effects creator-driven social platforms, with Alexis Ohanian, cofounder from Reddit, Paul Davison, cofounder from Clubhouse, and a16z general partner Andrew Chen, whose new book, “The Cold Start Problem: How to Start and Scale Network Effects” comes out this week (see coldstart.com for more). We cover: how do you cold start and get your first creators? How does your relationship to creators change as you scale? And how is web3 changing the incentives and dynamics around network effects?

Highlights

  • What are network effects? [1:32]
  • How do you cold start and get your first users? [2:33]
  • Atomic networks and why minimum viable community is more important than minimum viable product [6:36]
  • How do you curate your network and set norms? [8:42]
  • Faking users: good idea, bad idea? [13:13]
  • What is flintstoning? [14:26]
  • How does the relationship to creators change as you scale? [17:07]
  • Building for the professional creator class [22:52]
  • How is web3 changing incentives? [25:12]
  • Paul Davison Paul is the cofounder of Clubhouse

  • Alexis Ohanian Alexis was a cofounder of Reddit and is now a business dad and an investor and mentor to entrepreneurs.

  • Andrew Chen is a GP at a16z in consumer technology. Prior to that, he led Rider Growth at Uber. He has written for a decade on metrics & growth on his blog/newsletter, and is the author of The Cold Start Problem.

  • Das Rush

Inside the GameStop Drama; U.S. Constitution, Auctioned

Ken Griffin and Marc Andreessen

Welcome to 16 Minutes, our show on the a16z podcast network where we talk about tech trends that are dominating news headlines, industry buzz, and where we are on the long arc of innovation. 

Today’s episode actually features a look back at the GameStop saga — the stock market drama that some headlines described as a “David-and-Goliath battle” that “upended Wall Street.” 

For quick basic context, here’s what happened: A group of Reddit users mass-purchased and drove up prices of stock in the video game retailer GameStop, forcing short sellers including hedge funds and institutional investors to back out in a short squeeze, pushing prices even higher. But beyond the news, this also portended other, broader trends including redefining the power of retail investors, the phenomenon of meme stocks, and more.  

So in this episode — which is from a conversation that originally took place live on Clubhouse  (and which, by the way, can also be found on the a16z Live feed) —  a16z co-founder Marc Andreessen talks to Ken Griffin, founder and CEO of the hedge fund Citadel, which was a key player in GameStop as both a market maker and investor. You’ll also hear a16z general partner and fintech expert Alex Rampell join later in the conversation.

Griffin also just purchased (in a Sotheby’s auction a little over two weeks ago) — one of the original copies of the U.S. Constitution, an auction in which a decentralized autonomous organization called ConstitutionDAO also bid on buying it; Marc and Ken discuss this briefly at the very end.

  • Ken Griffin

  • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

‘Play-to-Earn’ Gaming and How Work is Evolving in Web3

Arianna Simpson, Gabby Dizon, Jeffrey Zirlin, and Zoran Basich

In today’s episode we’re talking about an emerging model of gaming called play to earn, in which players can make actual money based on how much time and effort they put into a game. Play to earn is also part of broader trends — the changing relationship between players and platforms, new incentives for participants in blockchain-based networks, and the new internet era that is coming to be known as a web3.

The top play-to-earn game is called Axie Infinity, operated by a Vietnam-based company called Sky Mavis. Players of the game acquire unique digital pets called Axies, and battle other teams of Axies. These NFT Axies can be created and sold using the game’s in-game currency, SLP, which can be traded for traditional currency.

Think of it as Pokemon on the blockchain, with a social network built-in, and an actual economy, and even companies built around the game that help players onboard and loan them money to get started playing. The game has made more than $3 billion in total sales since launching in March 2018, with much of its early growth in the Philippines.

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.)

Our guests today are Jeff Zirlin, the cofounder of Sky Mavis; Gabby Dizon, the cofounder of Yield Guild Games, a play-to-earn-gaming guild that gives players the resources to start playing; and a16z crypto general partner Arianna Simpson.

They talk to a16z’s Zoran Basich about the larger tech trends that enabled the emergence of play to earn, why and where it caught on first, and the role of community, as well as the challenges, which include onboarding and scalability, and the economic sustainability of this model.

Transcript below:

Axie and the beginnings of play-to-earn

Jeff: This is something that took almost four years to marinate. So, a lot of the original Axie community members and founders were from the CryptoKitty community, where we saw that the production of new NFTs had to be related in some ways to actual work and effort. This would basically prevent hyperinflation. So in the first iteration of the Axie battle system, we had experience points and these experience points could be used to breed Axies. We then had the idea and a lot of our players were requesting that these experience points be tradable tokens. And so, we tokenized them.

And then we saw that our players had actually created a liquidity pool on Uniswap, which is a decentralized exchange for these tokens. We then realized that, hey, you can actually calculate an expected hourly wage for playing Axie because there’s guaranteed liquidity in this pool.

So, I think that was really the moment that we saw the potential for it. We put out a post in January of 2020 where we saw that there was this intersection of NFTs and DeFi that was creating something that we then dubbed play-to-earn, and I think that might be the first time that that terminology was used, at least the first time where it was used in a non-speculative way

Zoran: One key thing to note here is that people have to pay a significant amount of money to get started – they have to download crypto wallets and buy three Axies, which can cost upwards of $1,000, with prices fluctuating. So Gabby, you saw this from the ground in the Philiipines — what did you see in this economic model?

Gabby: So I’ve been in the game industry for almost 20 years and actually joined the Axie community in late 2018 as a player. And what I saw last year is that people from my home country in the Philippines had started discovering and playing Axie as a way to kind of escape the economic hardship of the lockdown. And these were people who weren’t crypto enthusiasts. These were regular people that were stuck at home that had no jobs.

So, came upon the idea of a guild as a way to scale the efforts to introduce people to the world of play-to-earn and provide Axies via a scholarship program that would provide access to people around the world, not just from the Philippines, but across Southeast Asia, in India, in Latin America, and other countries to be able to play the game and earn money from it without having to afford the assets upfront.

And what we do is that we kind of act as a player collective where we invest in a lot of these assets and then form the communities around these different games where people can participate into these networks so that it’s not anymore a hurdle to get into a game like Axie Infinity because I can’t afford three Axies.

The conditions for growth — the Philippines and beyond

Zoran: You touched on the growth in places like the Philippines. What were the conditions that enabled that? Why did that spark catch there? And what does that tell us about building communities in the future?

Jeff: I think that there are a couple of unique things related to the Philippines. The Philippines loves mobile games. There’s relatively high crypto literacy. It’s a very communal culture where information and trends can spread very quickly. Filipinos have traditionally been early adopters to many social networks and platforms like Facebook. With the Philippines specifically, I think there were a lot of really special circumstances that basically allowed it to break out ahead of the rest of the world, but I do think that it’s only around nine months ahead of the rest of the emerging market economies

How play-to-earn reflects broader web3 trends

Zoran: So if this model indeed spreads and continues to grow, that suggests there’s something bigger going on here – what is that, what transition are we actually seeing?

Arianna: We think that the people creating the value should be participating in the upside. And that’s really the core belief at the center of what’s happening with web3. People are earning their livelihood on Axie, and so having the ability to not have all the value retained by the platform, but actually, pass it back to the community, the creators, the people who, again, are really responsible for building the value is, I think, an unstoppable movement

Jeff: One of the frameworks that we can think about is, like, okay, who are the middlemen that are getting removed? And how is the value that was being extracted by those middlemen shared with the actual users and stakeholders of the platform? So with Axie, I think what’s happened is that the app stores and the game publishers have been removed from the equation. They traditionally take 50%-plus of the revenue that’s generated by a game. We’ve removed them. We don’t use them. And we’re sharing that value with the people who are actually driving traction for our network, which is the community.

How sustainable is it?

Zoran: Okay, people are paying to get in and they are interacting with their characters, and they’re competing, and they’re earning rewards. But help me understand the sustainability of that. How do you think about the underlying economic model here?

Jeff: So one of the ways to look at it is right now, Axie is a little bit of a growth-dependent economy, just like any emerging market nation. It is a little bit dependent on capital inflows. But long term, it’s really important for us to have players that are in the economy spending because they think that the game is really fun or that they see ways to trade, like, money for power or respect. And the more Axie becomes like a real social network, a nation, the more opportunities for those types of value exchanges to arise. And we have framed Axie as a social network as well as a game since the early days. And this is becoming more and more true as, you know, we’ve done things like max out our Discord.

Zoran: What are the key factors to making play-to-earn sustainable? Like, how do you think about potential ceilings on capital inflows and participation?

Arianna: In many ways, these economies are going to be similar to real-world economies. They’re very complex and I think continuing to manage the capital and token supply and all of the dynamics within these metaverses is going to be a real challenge.

One of the key things in my mind is making sure that there continue to be different groups of players who are participating for different reasons and deriving different kinds of value. So, people might be there purely because they enjoy playing the game or because they get a lot of personal fulfillment out of coaching others. In many cases, it might be financial. There are a number of different reasons why people might want to be playing the game and engaging in the community. But as long as the players who are putting in additional capital derive other kinds of enjoyment from it, that’s fine. And that’s actually not at all dissimilar to the real-world economy. I might be wealthier and I might pay someone to do something I don’t want to do and somebody might, in exchange, pay me to do something they don’t want to do.

And so as long as there continues to be a robust ecosystem of participants who are there for a variety of reasons, I think that really drives things forward.

Zoran: What is the balance between people who are trading within the economy versus taking money out of the economy, like, you know, transferring it to fiat, because people are paying their bills, right? And so they have a decision to make about how much to kind of put back into the system versus take out. What are the trends there? And how does that affect the business model?

Jeff: In terms of the capital flows, we see that there are far more funds that are being deposited into our ecosystems than withdrawn. Only 4% of Axies are for sale on the marketplace. So, I think this shows the emotional connection and the fact that people don’t just see this about money. It’s also something that is giving them access to new opportunities, social networks, and is a lot of fun

So, right now, I think, it’s easier to balance the economy when we’re growing really quickly and.the animal spirits are active. Long term it will be about continuing to make sure that the community is insanely fun to be a part of and the game is insanely fun to play.

The role of community

Zoran: But what is it that keep people feeling as though they’re part of a community? There is some bigger, evangelizing feeling here than just, “Yeah, I’m gonna make a few bucks.” Right?

Jeff: Sure. I think it boils down to this shared economic alignment, but also cultural alignment. This community has been around since before NFTs were popular and, you know, what everyone in the world was interested in. Right? So, this exploration, I think that’s part of the DNA of the community. And as new entrants come in, they learn about this and they actually have some of the ideals transferred into them as they join. So, I think that’s been really important. And the community is amazing at the education and the onboarding. Eighty percent of our users are coming from referrals.

Arianna: I think it just demonstrates the enthusiasm that people have for what’s happening, both in physical space, by the way — the meetups have incredible attendance, people are really organizing themselves on the ground — but also, of course, in the digital realm, so in Axie itself, in the Discord, in the Substacks, etc. So, there’s just a stickiness that comes from people feeling like this community is theirs and they are benefiting from being members of it rather than, you know, having someone extracting value from them.

Zoran: But it seems like there’s kind of a paradox here, right? Because one of the things about legacy systems, whether it’s financial systems or gaming, is that they’re very sticky. It’s hard to move your account from one bank to another or you can’t move your in-game goods from one game to another. And crypto has kind of changed that and it’s made it much more portable. Isn’t it easy for some other game to pop up that is just as appealing and the characters are just as cute and your community might migrate over there?

Jeff: What we’ve built is not just the gaming community, it is, in many ways, a nation where people have shared cultural values, there’s overlaps and entertainment, there’s even this lingo or jargon similar to language. We have this very deep economy. So, I think it is much harder with this type of network to uproot a community and transplant them into a new universe where they don’t have a stake in it. We’re seeing people in the real world form Axie communities where you know all the people in your town or your city who plays Axie Infinity. And it has network effects, right? The more people that own Axie tokens, the more people that own Axies, the more people that own land within the universe, the deeper entrenched these economic and social relationships get with each other.

Gabby: Yeah. One of the hallmarks of the shift between web2 to web3 is that the communities are opt-in and they’re incentive-aligned by shared economic ownership, by the kind of traits that lead people to share the same affiliation, the same tribe around maybe certain assets or certain game universe. People stay here because they choose to be here, and they help build the culture of a tribe. There’s a shared economic incentive, there’s a cultural incentive. But if people want to quit that network and leave and join another network, no one is preventing them to do so.

Zoran: So speaking of user choices and user behavior, how is that changing? Because YGG is funding players, bringing them into the game, helping them get started, you’ve got a close-up view of this – is there a pattern of behavior, or different modes of participation that are evolving?

Gabby: We see ourselves as a necessary layer to bring people from the real world into the metaverse, especially for those that can’t afford it. But once they’re there and have an income, we actually encourage them to turn people from gamers to investors.

So, the first few cycles of people are earning money via SLP. They sell it for fiat, put food on the table, pay their bills. What we see is that when they’re a few cycles in and they have some excess income, many of them for the first time in their lives, the behavioral patterns actually change. People have more of an investor mindset. And now they think about, “Do I buy Axies for myself and graduate from this program so that the next person can benefit from the Axies that I was using? Do I buy Axies for my family members so that they can start earning money too? Do I buy my first piece of land in Axie Infinity and become a virtual landowner?” And so on and so forth.

Scalability challenges and Ronin

Zoran: Jeff, you mentioned CryptoKitties earlier. That’s the most prominent example before this of a game that really reached some level of mainstream success. And it kind of brought down Ethereum for a while, right? So in terms of growth and scalability, what are the challenges?

Jeff: In our ecosystem we have a lot of small, kind of low-value transactions that are really key to our user base in the emerging markets. And these were things that were basically priced out on Ethereum, where on Ethereum they might be $5 to $15 and obviously we’ve been in a bull market and that’s also coincided with rising gas costs on Ethereum as well. We were at the mercy where the success of the Ethereum ecosystem and all these DeFi applications was actually strangling out our growth, right? It was pricing our users out. We were in a situation where we really needed to migrate the majority of our transactions onto our own infrastructure.

It’s really this April with the launch of Ronin, which is our Ethereum sidechain, where we added that key piece to the equation, which allowed for our growth. We were around 38,000 daily active users in April before the launch of Ronin. And we just hit 2.4 million daily active users.

Zoran: What exactly did Ronin, this sidechain, do for you — what did it unlock?

Jeff: It allowed for the proliferation of the scholarship model, because there are a lot of transactions that are involved in running a scholarship program. So, breeding Axies, sending Axies to different accounts so that they can be distributed to scholars, claiming in-game rewards, these are all transactions on a blockchain that on Ethereum would cost a lot and, oftentimes, take a while, whereas on Ronin, these have all become very cheap, free up until now, and much faster.

That’s why we call Ronin Ronin. Right? Ronin is the samurai without a master. This was all about taking our destiny into our own hands where we could be the ones that determined our growth path rather than having to be a taker of market conditions and gas prices on Ethereum.

Arianna: I think that just really tells a critical story. If they hadn’t done that, I think the game would have obviously continued to grow, but been capped in the way that it was before. And so once there was that infrastructure put in place, there was an incredible unlock, which is really visible from the chart.

Onboarding into the game

Zoran: And so the other challenge too is onboarding, in terms of the steps required to start playing the game. You have to download multiple wallets and you have to, obviously, pay some money. And on the one hand, that shows how appealing the game is for people that they are willing to take these steps. But on the other hand, you want it to become easier.

Jeff: It is still incredibly difficult to get started with Axie. So, a lot of our development roadmap is aimed at reducing these barriers, specifically, for example, right now, if you are interested in Axie and you want to play, you then have to figure out which Axies you actually want to buy. This might involve doing a lot of research on the internet, maybe watching YouTube tutorials. And you’re basically buying characters for a game that you’ve never played before.

We will be releasing an upgraded battle system. And one of the features of that will be kind of this demo tutorial where everyone will be able to download Axie, get a free team of starter Axies, learn about the game, figure out if they actually love the gameplay, love the community before they have to make any economic decisions. So, we think that that’s going to be a really important stage between awareness and activation.

There are also payment on-ramp and off-ramp frictions that we can work with partners on to also improve that onboarding experience.

Gabby: And, you know, That’s where the community steps in. And what I love about it is that you’re replacing the middlemen such as Facebook and Google with community-based structures that onboard people into games like Axie, teach them how to play the game, how to earn money, how to use crypto, onboard the wallet. And for me, it’s like web2 reduced people into statistics that it’s just about daily active users. And with web3, with this community-based acquisition growth returning them back into individuals, again, individuals who are like creating content, learning how to play, people who are earning money, and we see all of these stories of people whose lives were profoundly changed by earning money And I think that’s really significant.

Play-to-earn and the future of work

Zoran: The idea of work is really important here. I mean, so far, it’s players, and people funding those players. But when you squint your eyes a little bit into the future, what kinds of new jobs do you foresee, whether it’s specific jobs or categories of jobs?

Jeff: There are many different archetypes of Axie players, right? There are the competitive battlers. There are the collectors. There are the scholarship managers, as well as the scholars who are, you know, primarily kind of farming tokens and selling them within the ecosystem. There are just people who are Axie holders, there are the content creators and the educators. And many players see themselves as more than one.

I think what we’ll see, within at least the Axie ecosystem, is that the different types of gamers will start to correlate or map to different professions, right? So, there might be someone who specializes in, you know, creating consumables, like a potion maker. Might be like, I don’t know, the version of a pharmacist. You’ll have your gladiator which might be, like, similar to, your athlete. So, we’re starting to see the rise of the competitive Axie players who are able to live off of, you know, winning tournaments and climbing the leaderboard.

I think that we’ll also see politicians arise in the Axie universe — people who are leading committees, for example, and thinking about the best use of funds, that might be the treasury, that might be the ecosystem, that might be, like, putting forth governance proposals. It might be creating requests for funding for different initiatives. You might have people who are focused on accumulating or harvesting certain materials. And that might be like the version of a farmhand. Right? So, someone who’s going on to Axie land, for example, and harvesting resources. I think that’s, like, an archetype that we’ve seen in the past, but I think has never really truly broken out and I think it’s had its breakout moment with Axie and the rise of YGG and similar institutions.

Zoran: We’re hearing a lot about web3 and the metaverse, and other buzzy phrases, all kind of revolving around this idea of more and more of our lives being lived online, in increasingly deep ways. Some of these trends we’re talking about with play-to-earn, do they go beyond games, or even jobs, and just become part of how we live?

Gabby: Yeah. This intersection between crypto and the creator economy, it’s actually one of the things that excites me the most about the future of work and where people are going in the metaverse. So, when you see these games in virtual worlds, I think it will not be just in games like what we’re seeing now. There will be virtual worlds where people can be, for example, going to the bank, interacting with a DeFi application, and doing what we just think of as work but they’re doing this in these virtual worlds that may look like games, but they’re actually just the interface for doing this type of work in the virtual world. So, I think more and more the blending of work and play will come together and we’ll be using crypto as a means to exchange value, but they may not necessarily be games as we know them today.

Gaming as a pathway to crypto adoption

Zoran: And I wonder what you think about crypto adoption overall —there’s still a fair amount of skepticism by a lot of people. What does play-to-earn and gaming, in general, mean for crypto adoption?

Arianna: Gaming is going to be a key way in which the next hundreds of millions of users onboard into crypto because, historically, you know, it hasn’t been easy to get involved and the barrier to entry is pretty high, the technical hurdles are fairly high. And it can be a little bit scary. There’s real money at stake. And what games offer is a much friendlier, more approachable onboard. You have these cute digital pets. You can go on and meet other players. We’ve talked about a real sense of community. And so it’s just a much more approachable way to start playing around in the space. And so we see that players might come in through a game and then, you know, once they have a wallet, start to experiment with other web3 products and experiences, and really expand outward from there.

Zoran: And how much of the crypto will end up being under the hood in the future? Will people kind of be transacting and not even thinking about it as crypto, but, you know, buying things with credit cards? Tell me how you think about that.

Jeff: What we need to do as product developers is to make sure that our user base and our prospective users really understand what the benefits of our products are, right? The benefits are derivations of blockchain technology, I don’t think they need to understand, like, how blockchain actually creates these benefits. So, I think when they see a fun, cute game where they can actually earn real value, I think that’s good. That’s sufficient. And I think that’s a strong enough pull. I think that learning how to use blockchain, I think these are more emergent benefits that they might get. Once they get involved, you know, they might kind of fall down the rabbit hole. We love seeing our user base go through that process as well. But I think in terms of the awareness phase, the benefits should be crystal clear. Even though the onboarding is difficult, people and organizations are onboarding by hand the next generation of blockchain users.

Zoran: Arianna, Jeff, Gabby, thank you so much for being with us today.

Gabby: Thank you.

Jeff: Thanks for having us.

Arianna: Thanks, Zoran.

  • Arianna Simpson is a general partner at a16z crypto. Prior to joining, Arianna founded Autonomous Partners, an investment fund focused on cryptocurrencies and digital assets.

  • Gabby Dizon is the cofounder of Yield Guild Games, a play-to-earn gaming guild. He is an 18 year veteran of the game industry and has been in the blockchain game and NFT space since early 2018.

  • Jeffrey Zirlin is the cofounder of Sky Mavis, where he spearheads the growth of products through messaging, community empowerment, and incentive design.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

16 Minutes: Steam Halts Web3 Games; FDA Approves Prostate Cancer AI

Jonathan Lai, Eddy Lazzarin, Vineeta Agarwala, Jay Rughani, Eliezer Van Allen, and Zoran Basich

Welcome to 16 Minutes, our show where we talk about tech trends in the news! We have two segments today:

1) The announcement recently that Valve Software, which operates the massive gaming platform Steam, added a rule barring games that use blockchain technologies or that allow users to exchange cryptocurrencies or NFTs – this rule appeared on its “What you shouldn’t publish on Steam” onboarding list for developers. We go beyond the players to the trends at play here, putting the news in context — as is the premise of this show — because it not only immediately impacts gaming developers and gamers using the platform, but has implications for gaming business models and the arc of innovation in gaming as part of the web3 movement.

Our expert guests are a16z partner Jonathan Lai and a16z partner Eddy Lazzarin.

2) The FDA’s announcement last month that it authorized marketing of the “first artificial intelligence (AI)-based software designed to identify an area of interest on the prostate biopsy image with the highest likelihood of harboring cancer so it can be reviewed further by the pathologist if the area of concern has not been identified on initial review.” The FDA reviewed the technology from Paige Prostate through its De Novo regulatory pathway.

We have three expert guests: Eli Van Allen, associate professor of Medicine at Harvard Medical School and chief of the Division of Population Sciences at the Dana-Farber Cancer Institute; a16z bio general partner Vineeta Agarwala; and a16z bio partner Jay Rughani.

  • Jonathan Lai is a partner at a16z where he focuses on games, social, and creator economy investments. Prior to joining the firm, Jon led North America investments at Tencent and was a PM at Riot Games.

  • Eddy Lazzarin is head of engineering at a16z crypto. Prior to that Eddy was a software engineer at Netflix working on data ingestion systems, and data engineer at Facebook working on growth analytics for Messenger.

  • Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

  • Jay Rughani is a partner at Andreessen Horowitz investing in bio and healthcare technology companies. Prior to joining a16z bio, he worked for Flatiron Health, which was acquired by the Roche Group in 2018.

  • Eliezer Van Allen

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Crypto Security and the New Web3 Mindsets for Users

Eddy Lazzarin and Zoran Basich

Today’s episode is all about crypto security — that is, the new mindsets and the new strategies for storing crypto assets safely while also allowing holders control and access. 

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.) 

We’ve covered security trends more broadly a ton in our content, which you can find at a16z.com/security, as well as crypto-related trends including NFTs, and the creator and ownership economies.

But as more people enter crypto lately — thanks to the boom in NFTs, decentralized finance, and much more — we share specific best practices and options for securing crypto as well as discussing how it all fits this next evolution of the internet: web3.

Our expert today is a16z crypto data scientist Eddy Lazzarin, who joins host Zoran Basich. He covers practical approaches ranging from passwords to crypto wallets and what users can do; the evolution of crypto briefly; and the big picture mindset shifts involved here as well.

  • Eddy Lazzarin is head of engineering at a16z crypto. Prior to that Eddy was a software engineer at Netflix working on data ingestion systems, and data engineer at Facebook working on growth analytics for Messenger.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Cloud Wars & Company Wars: Play Nice, But Win

Michael Dell, Marc Andreessen, Martin Casado, and Sonal Chokshi

    There are lots of challenges in being public while trying to innovate, and limits to being a private company as well; but it’s rare to see a company go public then private then back to public again. As is the case with Dell Technologies, one of the largest tech companies — which went private 2012-2013 and then also pulled off one of the most epic mergers of all time with Dell + EMC + VMWare 2015-2016 (and which we wrote about here at the time).

    Is there a method to the madness? How does one not just start, but keep, and transform, their company and business? Especially as it adapts to broader, underlying tech platform shifts. Michael Dell shares all this in his upcoming new book, Play Nice But Win: A CEO’s Journey from Founder to Leader… he also, tellingly, may be one of the longest-standing founder-CEOs (37 years so far).

    Because this is really a story about innovation, who decides, who judges, who does it, and where: In the markets, in public, in private; in the both the big picture and the inner detailed workings of a business beyond “cells in a spreadsheet”; and even in fighting — or harnessing! — narratives, whether it’s the demise-of-PC or cloud wars 1.0 /2.0… And where trends like the cost paradox of cloud, and “end of cloud” edge computing, among others like AI & ML, also come in. In this special book-launch episode of the a16z Podcast with Marc Andreessen, Martin Casado, and Sonal Chokshi debate the Cloud Wars to the Company Wars (along with some behind-scenes stories and even some star wars) with Michael Dell… and whether you can really play nice to win.

     

    image: Dell EMC World 2016/ Dell Inc.

    • Michael Dell

    • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

    • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

    • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

    Uncontrolled Spread: Science, Policy, Institutions, Infrastructure

    Scott Gottlieb, Vineeta Agarwala, Marc Andreessen, and Vijay Pande

      There’s no question technology played a huge role in the recent/current pandemic, including especially in the plug-and-play engineering and incredibly fast development behind the mRNA vaccines… But is there an even bigger role for the private sector, not just government, to play (and partner) when it comes to key infrastructure for future such emergencies, and even beyond?

      Especially given how faulty the translation of institutional science to policy and public health measures turned out to be — for instance, with “6 feet” of social distancing, or with fomite (vs. aerosol) transmission of COVID. And why are we still talking about the same, not specific, vaccine booster for the Delta variant? What can we learn about real-world evidence, other clinical trial approaches, and progressive (vs. binary) EUA approvals when it comes to public health emergencies? Are capabilities like genomic surveillance and mapping strains — which require layers of technology, real time — sitting in the right places?

      In this special book-launch episode of the a16z Podcast, former FDA commissioner Dr. Scott Gottlieb — author of the upcoming new book, Uncontrolled Spread: Why COVID-19 Crushed Us, and How We Can Defeat the Next Pandemic — shares insights on the above, and revealing stories from behind the scenes. Do we need a new entity to manage public health through a national security lens, and is the government capable? Gottlieb debates this and other probing questions from a16z co-founder Marc Andreessen (who famously wrote “It’s time to build“); a16z bio general partner Vineeta Agarwala MD, Phd (who has spoken about the trials of clinical trials, practiced medicine during the pandemic, and more); and founding a16z bio general partner Vijay Pande PhD (who, among other things, founded the distributed computing project Folding@Home which pivoted to COVID proteins).

      One thing’s for sure — with this COVID crisis, we’re at an inflection point between old and new technology — whether it’s in how we make vaccines, or how we apply the fields of synthetic biology and genetic epidemiology in public health response. So now’s the time to look both backward, and forward, to really change things…

      • Scott Gottlieb served as the 23rd Commissioner of the U.S. Food and Drug Administration (FDA) from 2017-2019; is a resident fellow at the American Enterprise Institute, partner at NEA, and serves on several boards.

      • Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      Man, Mosquito, Malaria Vaccine

      Jorge Conde, Rajeev Venkayya, and Sonal Chokshi

      Playing out against the backdrop of a global pandemic (including recent massive surges in regions around the world) is the news that came out a week ago that a candidate “malaria vaccine becomes first to achieve WHO-specified 75% efficacy goal”. While the findings are still in preprint with The Lancet, the resulting buzz and phrases quoted included everything from “unprecedented”, “groundbreaking work”, and “very exciting” to “high expectations”, “highly effective”, and “a hugely significant extra weapon”… A “weapon” in the war against malaria that is — a disease that is estimated to cause over 400,000 deaths each year globally, and predominantly in children under the age of five.

      So in this special 2x explainer episode of 16 Minutes (also running on the a16z Podcast), we —Rajeev Venkayya of Takeda Pharmaceuticals, a16z bio general partner Jorge Conde, and Sonal Chokshi — dig into what’s hype/ what’s real about this news, beyond the headlines and beyond the buzz. What does the data tell us, what does the current study phase mean, and what’s left to get to widespread, real-world use? How does this candidate vaccine (R21 from Jenner Institute/ Oxford University) compare to the other malaria vaccine (RTS,S from GlaxoSmithKline)? How do, and don’t, advances in and around COVID vaccines play here? And why has it been so hard to develop vaccines for this particular disease?

      Because we also cover (as is the premise of the show) where we are on the long arc of innovation… and this is an innovation story that’s been nearly a century in the making.

      Show Notes

      • The urgency of finding a vaccine to combat malaria [2:45] and why malaria is so difficult to treat [5:12]
      • Discussion of the R21 vaccine candidate and its pending study [10:48]
      • Previous vaccine attempts, details around clinical trials [17:48], and the pros and cons of different vaccine approaches [24:27]
      • How vaccine effectiveness is measured [28:54] and possibilities for the future [33:35]

      Transcript

      Sonal: Hi, everyone. Welcome to the a16z Podcast network. I’m Sonal, and this is a special 2x episode of 16 Minutes, which we’re also running on the main a16z Podcast as part of our ongoing such coverage — and it’s all about the recent news, science, technology, problem, and innovations behind malaria vaccines.

      While the discussion includes contrasts and comparisons to COVID vaccines briefly – and also plays out against a broader backdrop of the massive recent surge of cases in India and South Asia, as well as new waves in Brazil, Turkey, France, Argentina and elsewhere — the big news that also came out this past week is that a new candidate malaria vaccine is the first to achieve the World Health Organization’s goal of 75% efficacy, according to the announcement from Oxford University

      Just to give a sense of how big and buzzy this news has been, some of the keywords that have been quoted by experts in many of the releases and articles have included phrases from “unprecedented,” “groundbreaking work,” “very exciting,” to “high expectations,” “highly effective,” and “a hugely significant extra weapon.”

      So, as is the premise of 16 Minutes (which, if you’re not already subscribed to, be sure to find, and follow, in your podcast app), we dig beyond the headlines for what’s hype/what’s real, as well as where we are on the long arc of innovation.

      Our expert guests for this episode are Rajeev Venkayya, President of the Global Vaccine Business Unit at Takeda Pharmaceuticals, where he leads full-stack development of vaccines for tropical diseases like dengue, norovirus, and Zika. He’s also been trained as a medical doctor and served as Director of Vaccine Delivery at the Gates Foundation, and was previously at the White House in biodefense.

      Rajeev has also shared how vaccine development works in general, including outlining the phases and what was accelerated for COVID vaccines in an episode the three of us did last year, with a16z bio general partner, Jorge Conde, who also joins this episode, and has, in fact, been on all of our vaccine episodes. I don’t know if you actually knew that Jorge, that you’ve been on every single vaccine episode. <Jorge: You keep inviting me back!> Yeah, well, sadly — and importantly — we’ve had to cover many different aspects of this topic over the past year and a half.

      And, we’ve covered everything from vaccine nationalism and vaccine hesitancy to vaccine manufacturing and scaling and all about mRNA vaccines and much more. Listeners, you can find all of that at a16z.com/vaccines.

      But for this malaria vaccine, I’d actually love to start by hearing from both of you what your reactions were to the recent news. And I’ll summarize the specifics of the news in a moment, but, would love to just quickly hear the big picture for why this matters, from your vantage point. 

      The fight against malaria

      Rajeev: I think this is a really big deal, because malaria is one of the big three (as we call it) — HIV, tuberculosis, and malaria cause an extraordinary amount of suffering and deaths every year.

      All three of them have proven to be very, very tough targets from the standpoint of vaccine development. There’s no vaccine yet for HIV, despite extraordinary global efforts for decades. We have a vaccine that leaves a lot to be desired in the BCG, that we give at birth for tuberculosis. And there is a malaria vaccine that was brought to the world a few years ago from GSK called RTSS — but, that vaccine, while efficacious, has some room for improvement, and this vaccine potentially could be that improvement we’re looking for.

      Jorge: In addition to what Rajeev said, I would say two of my biggest reactions was number one, the fact that they seem to have early indications of a vaccine that’s highly efficacious against a parasite is no small feat in and of itself (a parasite is a tricky bug).

      And the second one I would just point out is, you know, obviously, vaccines have gotten an incredible amount of attention over the last year or so, given the COVID pandemic. But this vaccine, first of all, has been decades in the making… <Sonal: Nearly a century, I heard!> …nearly a century of effort to try to find a vaccine against this parasite, and this breakthrough didn’t come from the same technology that gave us the COVID vaccine breakthroughs. And to me, what’s fantastic to see is, the dividends that come from decades of research that pay off using, sort of, “old-world” technology. What I mean is, you know, this was a vaccine developed using more traditional vaccine production methods, versus what we saw in the course of one short year with the mRNA vaccines that BioNTech and Pfizer and Moderna brought us for the COVID pandemic.

      Rajeev: Yeah I think for a long time to come, we’re gonna think about vaccines in terms of “pre-mRNA” and “post-mRNA.”

      This is technology that was developed well before mRNA vaccines came of age. The technology that was used is actually proven — it’s been used in the hepatitis B vaccine, as well as the human papillomavirus… <Sonal: HPV, yeah> …yeah, these virus-like particles that are very, very effective at “presenting antigen” to the immune system so that it can recognize it, and then develop an antibody and cell-mediated immune response to that antigen.

      The challenge of finding a vaccine

      Sonal: Well, let’s dig into what this vaccine is and how they work. But first, just to put this whole problem in scope, just quick statistics. Over 229 million cases of clinical malaria were reported the year before [2019], and the World Health Organization estimated that malaria causes over 400,000 deaths each year, globally. I mean this seems to be concentrated in Africa and South Asia, Southeast Asia, different regions, but it is important to just note the scope of the disease.

      Jorge: A lot of these tropical diseases also happen to be “poor country diseases.” I have to wonder if this tropical disease was endemic in the richer parts of the world — had we, you know, gotten to this breakthrough sooner.

      Sonal: I was wondering the exact same thing, Jorge. The fact that we got to a COVID mRNA vaccine in less than a year, versus this disease that people have been working on for nearly a century — it almost makes you wonder, like, if this were prevalent in the United States, we would have solved this like 20 years ago.

      Rajeev: Possibly, possibly. There would have been a lot more R&D for sure.

      Sonal: I frankly think this is also true for women’s health, but I will go on that rant later.

      Jorge: I was about to say that.

      Rajeev: Yeah. It’s important to note that almost all the deaths are happening in sub-Saharan Africa, and two-thirds of those deaths are in children under the age of five. This is a very, very significant global burden of disease that desperately needs a safe and effective vaccine.

      Jorge: What some people don’t realize is, not only is the disease burden high, but malaria is something you can get infected with again and again and again. You don’t get sick once and then you’re immune. There are reinfections. And so that just adds to the burden.

      Sonal: I’m really glad you’re bringing up the point that it’s not a one-and-done disease, like another type of disease. I actually read a statistic that the average in a lifetime is six times, that a person can get malaria six times in their lifetime.

      Jorge: At some point, you do develop immunity. Like, the reason why people catch malaria an average of six times and not an average of the number of years that they’ve lived — because eventually you become more resistant to it.

      Sonal: Right. It’s like my parents, they never get stung or sick. But when we were kids, my brothers and I used to count and compare our mosquito bites, and we’d have, like, competitions for, like, how many we had. I would have (I’m not exaggerating), hundreds of bites on my body. And we did everything by the way. Nets, the coils, everything, you name it. The anti-malarial drugs — although I didn’t take mine, which is why I got sick. I was a young kid. I spit mine out. It was disgusting, and no one watched to see if I swallowed it.

      Jorge: And by the way, the way that these anti-malarial pills work (at least some of them), is they essentially poison your blood so that the parasite…

      Sonal: They taste poisonous. They taste like poison.

      Jorge: Yeah, the parasite can’t survive in your bloodstream.

      Sonal: So, why has this disease been so difficult? And in general, is there a difference when you’re designing a vaccine to target a parasite versus a bacteria or other viruses? The reason I’m asking is because, Jorge, you mentioned the malaria parasite — it has a complex life cycle, and can mutate. <Jorge: It’s wild> It’s crazy. I mean I’ve experienced a very mild form of it when I was a child, and just a little anecdotal bit of detail (that I don’t know if people who have never experienced this would ever know this), but in my experience, I had like the highest fever one day; the next day, it was as if I were a completely healthy person. And then the third day, I had a super high fever again. I don’t know if that’s a normal thing, but that was bizarre to me. Like, I had no idea why that even happened.

      Jorge: Well, part of the reason why that probably happened is this sort of funky lifecycle of the parasite. So, you know, an individual gets infected when they are bitten by a female mosquito that’s carrying an infectious form of the parasite. That goes to your liver, where it continues to reproduce. Then it gets released to your bloodstream, where it attacks your red blood cells. And, it replicates within your red blood cells, and, when your red blood cells get too full of parasite, they burst, release a bunch of parasites into your bloodstream — that’s what causes the fever to spike. And then, it gets tamped down, and then when there’s another burst (of another set of red blood cells), the fever spikes again.

      And then, to complete the life cycle, you now have a sort of premature version of the parasite floating around in your bloodstream, and you get bitten by a mosquito — and now that goes back up into the mosquito (to complete its sexual maturation). So it actually comes full circle. A lot of people think about parasites having a host. You know, in the case of malaria, malaria is being raised in sort of like shared custody between man and mosquito.

      Sonal: That’s an incredible explanation.

      Do you have any thoughts specifically on what it means to target malaria, as a disease?

      Rajeev: Well, it’s almost a self-fulfilling situation, because the fact that a person can have malaria multiple times tells us that the immune system is having a tough time with this parasite. The immune system is not able to identify the parts of the parasite that it can then attack when it gets reinfected to prevent the illness from recurring.

      And so, if the immune system, which is super sophisticated, is not able to do that, then almost by definition, it’s going to be a tough vaccine problem. One of the reasons is, the parasite can be quite effective at evading the immune system in the way that it grows in a person’s body. The parasite’s life cycle involves transmission through a mosquito that bites a person who gets infected — but then once that parasite is in a person, it has multiple stages of its growth that can be difficult to target. And so, that’s another unique feature of malaria.

      A promising vaccine candidate

      Sonal: So, let’s talk specifically about this vaccine. To summarize, the candidate vaccine here, it’s called R21. It was developed by scientists at the Jenner Institute at Oxford. And the reported findings are that they demonstrate a high-level efficacy of 77% over 12 months of follow-up, in a study with African children. Specifically, 450 participants aged 5-17 months, all from the country of Burkina Faso. Most of the doses were administered before the peak malaria season there (three vaccinations were administered at four-week intervals), and then a fourth dose came one year later.

      So, that’s a super high-level summary. One more quick note, this was all part of the phase 2b trial — randomized, controlled, double-blind — the findings are still in press with the medical journal The Lancet (and I will include all links and sources that are mentioned in the show notes as always).

      Rajeev: One thing I do want listeners to know is that the data we’re discussing today and you’ve seen in the media comes out of a preprint — which means this is not yet a peer-reviewed publication. Other experts will look at the study design and the results and they’ll ask critical questions of the researchers that they’ll then have to address in their responses, ultimately resulting in a peer-reviewed publication. Now of course we do hope that the essence of the findings will remain unchanged, and that the conclusions will largely be the same — but we can’t say that for sure until the peer review process is actually concluded.

      Sonal: I mean, the preprint — lately “science-by-press release” is definitely a thing that’s accelerated in the last year for sure. I’ve seen it at a whole new scale that I’ve never seen before.

      Rajeev: Yeah, and you know some would say that given that we’re living in COVID times, where literally every day matters in science, that we have to accept science-by-press release, hopefully followed very quickly by peer-reviewed publications. But the peer review and publishing process just takes too long, frankly, for COVID. And so we often end up having to rely on press releases and preprints.

      But, I do want to point out that the preprint came out right around World Malaria Day, which is April 25th. And I think that’s probably why these guys released the preprint when they did.

      Jorge: Wait, it’s April 25th you said? <Rajeev: Yeah> So World Malaria Day is the same day as DNA Day. That’s interesting.

      Sonal: Oh, that is interesting!

      Rajeev: I didn’t realize we observed DNA Day. That’s…

      Jorge: Yeah, because April 25th was when the Nature article [was published] that Rosalind Franklin published her Photo 51, and Watson and Crick published their one-page, structure-of-DNA paper. And then, in an act of symmetry, the tie between Francis Collins and Craig Venter in sequencing the human genome was announced by Bill Clinton on April 25th.

      Rajeev: Oh my gosh, I didn’t realize we were — there was a World DNA Day. Thanks for sharing that.

      Sonal: So, back to the point about malaria. Anything more to say on the specifics of malaria as a disease that’s relevant here?

      Rajeev: Well, there are a couple of things to think about when you are looking at vaccine development. And one reason, and perhaps the most important, is to prevent the severe illness that comes from it. But there is another objective with malaria vaccines, which is blocking transmission. Now, we’ve all heard with COVID, that one of the goals of vaccines is to reduce transmission in the community to help us get a handle on the pandemic. The same concept applies when it comes to malaria, but the vaccine approach is very different.

      And — this will blow your mind — when we think about blocking transmission of malaria, the way it’s being approached from a vaccine standpoint, is to prevent a mosquito from picking up malaria from a person that has it. And the way you do that is by designing a vaccine that will generate antibodies that are taken up by the mosquito, along with the parasites, and preventing the parasite from reproducing inside the mosquito.

      So, you’re giving a person a vaccine that’s not going to prevent their illness, it’s going to prevent the parasite from reproducing in the mosquito, so you’re actually indirectly vaccinating the mosquito (and not the person), because that person could still get malaria illness.

      Jorge: The fact that we’re vaccinating mosquitoes, I think is wild.

      Sonal: That’s fascinating. Let’s actually talk about the findings.

      Rajeev: So, basically what the researchers did is, they took this group of children that are in a place with a very, very high incidence of malaria, right before the malaria season, and they gave them three doses of vaccine with another dose a year later.

      They then counted cases of malaria, so they monitored for fevers in the children, and when a child came in with fever, they would do a set of diagnostic studies, and if they were found to have malaria, then they would be classified as being a malaria case. And then you compare across the three groups, and the three groups were a control group that received the rabies vaccine, there was a low dose adjuvant group, and a high dose adjuvant group. And both of those had the same amount of the protein that makes up the vaccine.

      Sonal: Can you quickly explain what an adjuvant is and why it matters, because everyone always mentions that and how that plays in with the way the vaccine works.

      Rajeev: Adjuvants are what you might consider immune boosters. And so, for any given amount of let’s say protein that you’re giving somebody to train their immune system to recognize a virus or bacteria, or in this case, a parasite, you can get away with a smaller dose of that protein if you give somebody an immune-boosting adjuvant.

      The adjuvant that is used in this trial is the same adjuvant that a company called Novavax is using for its COVID-19 vaccine. And this is an adjuvant that is chemically related to an adjuvant that is used by GSK in their vaccine against shingles that is currently on the market.

      So, it’s an adjuvant that has been proven (at least in that vaccine) to be very efficacious. And at least based on the phase two data that we have, with the combination with this — what we call “virus-like particle” — it also appears to be quite effective at generating a protective immune response.

      Jorge: I understand why you’d have a high dose and a low dose arm. Why is the control arm a rabies vaccine versus saline?

      Sonal: Yes, I was wondering.

      Rajeev: Yeah, well, you know, when you’re looking at the immediate safety of a vaccine — meaning sore arms, fevers, or chills that you might get after a vaccine — you’re gonna see that to some degree with many vaccines. And so, we want to do a “fair or appropriate comparison.”. And so, using another licensed vaccine that would be appropriate for the population that’s in the study is an approach that’s often taken to have the control group even out between.

      Previous attempts and clinical trials

      Sonal: I see. You’re sort of controlling for the variables you’re trying to measure. <Rajeev: That’s right> That makes a lot of sense. So, why — this is the real question here, the big-picture question — why has it been so hard to develop a vaccine for malaria? Now, we’ve talked already about the difficulties of the disease, but, like, if you look at the arc of it — Rajeev you said there’s one vaccine I think right, the GSK one, RTSS, and I think the last thing that I read was that they demonstrated 55.8% efficacy in African children.

      Can you explain that vaccine, and tell us more about — I’m really trying to dig into, like, why it’s been so damn hard to actually get here, and why this milestone is so significant.

      Rajeev: Well, it was a big deal when GSK showed that their malaria vaccine worked a few years ago in a phase three trial. Now they showed that initially they had about 56% efficacy in the first year. Unfortunately, that efficacy wore off over time. And so, if you looked at how efficacious it was after four years, it dropped down to 36%.

      And so, that vaccine is not yet widely recommended. There’s a pilot program rolling out in a handful of countries that’s happening as we speak.

      Sonal: And by the way when you describe the pilot program, you were involved with that organization, Gavi (I believe) is the one that’s sort of helping the World Health Organization pilot the GSK vaccine in Kenya, Ghana, Malawi, I believe.

      Rajeev: Yeah previously, I served on the Gavi board, which is the primary financing entity for vaccines for low-income countries. The data coming out of that program will inform the future implementation of that vaccine. Based on that level of efficacy that was seen with that first malaria vaccine, the WHO later came out with a target efficacy of 75% for future malaria vaccines. That was what presumably these researchers were going for when they tested their vaccine, and they were actually able to hit that target.

      Sonal: So, the GlaxoSmithKline vaccine has been through lots of clinical trials — a lot more clinical trials than this one has — and this particular vaccine, going back to this specific news, of the Oxford vaccine news. This is a phase 2B trial, it’s not phase three yet. Can you quickly explain what it means to be in phase 2b and what comes next in phase three?

      Rajeev: Sure, sure. So when we take vaccines through clinical trials in people, we start out with the studies to assess the safety of the vaccine to pick up any significant problems in small numbers of people before you go into bigger trials. And that’s done typically in phase one. In those phase I trials, you’re also assessing what the right dose of the vaccine could be. So you might have high, medium, and low doses of the vaccine to give you a sense as to what the right dose is to take it to further clinical development.

      In phase II development, you’re often going into [a] larger number of individuals, and confirming that you are at the right dose. You might even still have different dose levels in your phase two. In a standard phase two trial for infectious diseases, you’re not actually looking to see whether you’re preventing infection, because there’s a relatively small number of people in phase two trials, and that wouldn’t be enough to actually be able to statistically measure a difference in the vaccine group versus the control group.

      A phase 2B trial is a little bit different: You have an even larger trial than a standard phase two, and these are often powered statistically to give you a sense as to whether the vaccine is actually working. This is a way that a company might de-risk the vaccine program before they go into a very large, very expensive, sometimes very long phase-three clinical trial.

      So, this is that type of trial, where given the high incidence of malaria in this region and this population, they were actually able to show whether or not the vaccine works at preventing malaria.

      The next step after this would be a large phase III trial. We can benchmark against GSK’s phase three trial of their malaria vaccine, which had about 15,000 children in it. And as we all know from COVID, the phase three clinical trials there have ranged from 20,000 to 60,000 individuals in any given phase three. So, this is smaller than that. And the main reason they were able to get away with a smaller trial is because the incidence of malaria was so high in the places where they were testing the vaccine.

      Sonal: And for specifics, what I understand is that the recruitment of the phase three trial has already started, and they’re recruiting 4,800 children aged 5-36 months across four African countries. But here’s a little twist. So, you mentioned Novavax — so they’re one of the partners that’s collaborating with Jenner. But the other is, of course, the dominant player, the Serum Institute of India. They are obviously going through a massive COVID — so they’re actually delaying this a little bit, from what I understand.

      Rajeev: Yeah, you know, it’s worth talking a little bit about that comparison between this vaccine and the GSK vaccine. Actually, they’re quite similar. They use similar adjuvants or immune boosters. The adjuvant used in the GSK vaccine is their proprietary adjuvant called AS01. In the case of the Oxford vaccine, it’s called Matrix-M, which is the Novavax vaccine. They’re both derived from tree bark (believe it or not) and so the chemical construct between the two, of the adjuvants, is quite similar.

      The other parallel between these two vaccines is the virus-like particle approach is the same one taken between the two. They both use the hepatitis B surface antigen, which forms the core (or the base particle) for the vaccine.

      The primary difference is that there is less of the hepatitis B surface antigen protein in the new Oxford vaccine. So there is proportionally much more of the malaria protein on the particle, than there is in the earlier GSK vaccine. And that is thought to be (potentially) a contributor to the difference that we’re seeing in efficacy with this vaccine, versus what GSK saw.

      Sonal: And to be clear — I just want to kind of take the bottom line on that. Basically, what both of these vaccines are doing is targeting the parasite in that sporozoite phase of the lifecycle, which is when it enters the human body from the mosquito. And, the vaccines you’re saying — the main difference between the two vaccines — the R21 includes a higher concentration than the GSK. But that’s the primary difference. Other than that, they’re relatively similar underlying mechanisms.

      Rajeev: That’s right, these two vaccines are quite similar insofar as they use a similar adjuvant, and they also have a similar structure.

      Different vaccine approaches

      Jorge: You know, in the case of this malaria vaccine, when we look at SARS-CoV-2/COVID as an example, the vaccine makers all sort of thought the spike protein was the best target or the most likely target, and that’s where the major players focused. Is there a similar consensus in malaria as to what the right targets are, or has that in and of itself been an odyssey?

      Rajeev: Great question. There’s a lot of consensus around what’s called the circumsporozoite protein, let’s just say it’s CSP — which is the protein that is used in both the GSK RTSS vaccine as well as the Oxford R21 vaccine that we’re talking about today. And given that we’ve had limited efficacy with the vaccines against CSP — it’s not to say that something else won’t turn out to be better in future vaccine clinical trials.

      Sonal: What happens if the bets we’re making with these vaccines — and this, I think, is the underlying thrust of Jorge’s question — is that it’s not actually effective at that sporozoite phase, do you have any thoughts on that? When we put all our bets on the coronavirus that hey we’re gonna focus on the spike protein, we’re taking a good bet — and so far it seems to have borne out, given that even with the new strains, that it’s still targeting the spike.

      Rajeev: Well, it’s possible that we won’t be able to get there with this protein as the primary target, and that we would have to perhaps add a second target to the vaccine in the future. And it’s possible that we would have to add a second protein, or go after a different protein, in order to get efficacy against the parasite in this stage of its lifecycle. There are a lot of other proteins that one could target on this parasite.

      Sonal: Got it. And actually, quick question for Jorge — is it naive of me to ask whether an mRNA approach would be more efficient? You guys put it really well, like, the old world/the traditional world versus a new world we’re in. Would a different approach to vaccines do a much better job? Because when I think of how we are thinking of these new batches of vaccines that we have in our bodies as, like, “software as a service,” are we able to do more with that — like, is that even on the horizon for malaria, or is that like just a pipe dream?

      Jorge: I don’t think it’s a naïve question at all. You know, I — the vaccine producers that have mRNA technologies are looking at a broad range of infectious diseases for their next areas of focus. So, influenza is — you know, the flu is going to be an area of focus. The common cold is potentially, you know, on the table. So, viruses like that, clearly are sort of a next-horizon focus for these companies producing mRNA-based vaccines, and I don’t think it’s unreasonable to assume that their focus will expand beyond that.

      I don’t know, technically, if an mRNA-based vaccine against malaria is feasible, but if you’re looking at surface proteins, arguably, you could theoretically develop an mRNA vaccine against the malaria parasite.

      Rajeev: I agree with what Jorge said. I don’t think we can say that just because mRNA is a new technology that it’s more likely to be effective against malaria; however, I do think we can say that we need to give it a shot, because there’s so many advantages of mRNA approaches relative to more traditional approaches of developing vaccines.

      One of the really interesting possibilities is that you could combine vaccine approaches into a single mRNA product. So, for example, you could have a CSP part of the sequence in your mRNA vaccine — which is the same protein that is targeted by these first two malaria vaccines that we’ve been discussing. And then you could have a second sequence or set of sequences that are targeting that sexual form of the parasite that the mosquito takes up when it has a blood meal, and then goes on to transmit the parasite to somebody else.

      Remember, one thing that we need to realize is that second vaccine target where you’re trying to prevent the mosquito from going on to transmit the parasite — a vaccine like that would not prevent the actual illness associated with malaria. So, if you had a vaccine that was just focused on that, you’d be giving it to somebody and telling them “Look, this isn’t going to keep you from getting sick from malaria, it’s going to keep you from passing the malaria parasite onto somebody else.” That’s not a very attractive vaccine for someone to take. But if you combine that with something that also prevents them from getting sick in the first place — which could be this other part of the mRNA vaccine — then you’ve got a vaccine that everyone’s gonna want to take, and helps us to reduce malaria transmission, and maybe even eliminate malaria long term.

      Measuring vaccine effectiveness

      Sonal: Wow. Okay. In general, for context, over 100 malaria vaccine candidates have entered clinical trials in the past decades. But none has shown this level of efficacy that’s been targeted by the World Health Organization.

      Now, again, to be clear, we’re talking about phase 2B. It hasn’t done large scale yet. But, one of the people who heads the World Health Organization malaria vaccine implementation program, argues that even modest efficacy would have a high impact precisely for the reason that people get the disease over and over again.

      So, can we quickly talk about what the numbers of efficacy mean? They evaluated the vaccine safety, efficacy, etc., over one year — and what it means in, like, real-world practice?

      Rajeev: Yeah, let me touch on the three parameters of vaccine performance that we often look at. One is immunogenicity. This is the easiest to measure. It’s simply the antibody response to the vaccine. We measure antibody levels in the bloodstream, and that becomes a measure of what we call immunogenicity. The reason that’s important is because the level of antibodies you generate with a vaccine might correlate to that vaccine’s ability to protect you from getting infected or contracting the illness associated with that infectious disease.

      A second term we use in late-stage clinical trials is efficacy. And what that typically means or measures, is the ability of a vaccine to prevent the illness associated with an infectious disease. That term “efficacy” is very specific to the context of a clinical trial. So, there is usually a point estimate (let’s say 70% efficacious), and it’ll have a confidence bound around that, which represents kind of the error range given the size of the sample of your study.

      The third measure we often talk about is effectiveness. Another way to think of this is real-world effectiveness. So, this is the assessment of how well the vaccine functions outside of a clinical trial, when you’re in the real world. You’ve launched the vaccine into a population (like we have with our COVID vaccines), and now we’re measuring how much illness and disease there is in a population.

      Outside of the very controlled environment of a phase three clinical trial, where you may be telling people to do a number of things in order to protect themselves from the infectious disease. And so, you might see that a vaccine performs very well in the context of a phase three clinical trial. Then once you actually roll it out to the population, it doesn’t perform quite as well because these are real-world circumstances where people aren’t doing all the same things they wouldn’t be in the context of a clinical trial.

      Sonal: So is there anything important to note about the question of effectiveness when it comes to this vaccine? Now again, it’s not in deployment yet, etc., but what are the considerations. One that of course comes to mind here is like, there’s like multiple doses.

      Jorge: Multiple doses, yeah, I think that’s the biggest one.

      Rajeev: That’s a big one. Multiple-dose is a big one. But there’s also, you know, in the clinical trial, people may have been very good about using bed nets at nighttime at home to prevent mosquitoes from biting. But in the real world, they may not keep up with their bed net use. They may not keep up with their indoor residual spraying of insecticides, which is often used to reduce biting in mosquitoes.

      So, there are a variety of things that could increase the likelihood of getting infected in a real-world setting, which could correspondingly reduce your measured effectiveness.

      Sonal: So, the other promise of this candidate vaccine from Oxford, is also the potential to get more high-volume, affordable vaccines. I read that it’s easier to make than the one that’s being used with the GlaxoSmithKline one, suggesting that it could be cheaper. Do you know if that’s true or not?

      Rajeev: I’m not aware of the differences in the cost of making the two, because they’re quite similar. But it is using a pretty standard method of making vaccines. It’s manufactured in yeast (which is a tried-and-true way of manufacturing protein) — and the fact that Serum Institute of India has done this, means almost best-in-class in terms of efficiency, plus very high quality of vaccine manufacturing, given Serum Institute of India’s great track record here.

      So, that’s also good news because this is a vaccine that is going to have to be priced at a level that is affordable for the poorest countries in the world, and can be purchased by Gavi. The fact that the cost structure is likely to be very low, helps to ensure that we’ll have a low price for this vaccine (at least for poor countries).

      Possible next steps

      Sonal: There hasn’t been a EUA for a vaccine in malaria, especially given the fact that malaria kills more people in Africa than COVID does currently. Do you think it’s possible that they might do some kind of EUA-type of situation for this?

      Rajeev: I don’t think so. I would expect them to go through a standard (although accelerated) review process. One of the things that you are able to do in a standard review process is make sure that the manufacturing processes are all very well worked out, and validated, and reproducible with high quality.

      It’s important for any vaccine, but certainly a vaccine like this that’s going to be going into vulnerable infants, you want to make sure that you do absolutely everything. And hopefully in the fastest time period possible.

      Sonal: Right. I mean, we don’t even have a vaccine for kids for COVID yet, in fact, right, so.

      Rajeev: That’s right, that’s right. One of the things that may be coming out of COVID is that we’re all a lot more attentive to global health problems that affect everybody in the world.

      Jorge: I have to believe that we’ve also now developed capability, capacity, and political will in terms of vaccine production to do these things at the scale and speed necessary to hopefully benefit the entire world. My hope is that not only have we developed new technologies — you know we are now in a post-mRNA vaccine world (as Rajeev mentioned) — that have the potential to be um pointed at other infectious diseases, and hopefully give us other future breakthroughs.

      When it comes to malaria, we’ve been pointing all of our guns at this for a long time. So, we’ve not only been looking for a vaccine, but as folks know, there have been philanthropic efforts to get bed nets out there. There’s of course tons of efforts in terms of insecticides, to reduce the population of mosquitoes. And there’s even, you know, engineering biology approaches to create genetically modified mosquitoes that are resistant to malaria. So, this is one weapon in what is a pretty deep armament to try to beat this thing.

      Rajeev: Yeah, it’s absolutely true that this could be a critical tool in our toolkit. I look forward to seeing the peer-reviewed publication and hope that we’ll be seeing just as good results when the vaccine goes into phase three.

      It’s also exciting to think about the end game. There is a day when we could imagine eliminating malaria from many more parts of the world, and possibly even eradicating it from the face of the Earth. Now, that’s not going to be easy, as smallpox and polio have proven (and certainly malaria is very different from those other diseases which are caused by viruses) — but it is something we can hope for.

      Jorge: I was gonna say, we know the date of eradication will be April 25th, we just don’t know what year.

      Sonal: Thank you so much, you guys, for joining this week’s episode of 16 Minutes.

      Rajeev: Thanks a lot for having me, Sonal.

      Jorge: Thank you, Sonal. Thank you, Rajeev.

      • Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

      • Rajeev Venkayya

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Creator Economy — NFTs and Beyond

      Kevin Chou, Jesse Walden, and Chris Dixon

      In today’s episode of the a16z Podcast, we’re talking about the Creator Economy, and how NFTs (but not just NFTs!) are making it possible for artists, musicians, videogamers, game developers, and writers to create entirely new markets to make money from their work and engage with their fans.

      Part of this emerging picture is social tokens, which share a crypto foundation with NFTs, but unlike NFTs (which are non-fungible tokens, in which each token is unique), social tokens are typically fungible, meaning each token has the same value. (Listen to our explainer episode “All About NFTs” with Sonal Chokshi, Jesse Walden, and Linda Xie, or see our curated NFT Canon for much more info on NFTs!)

      This hallway-style chat features a16z General Partner and crypto investor Chris Dixon, talking with Kevin Chou, who founded Kabam, and is the founder of Rally, an open network on Ethereum where creators can launch social tokens; and Jesse Walden, the founder of Mediachain, a music attribution protocol that was acquired by Spotify; he’s now the founder of crypto venture fund Variant.

      They’ll talk about how musicians, artists, and writers can think about NFTs and social tokens as well, and how those different types of assets can interact to create models that haven’t existed before.

      Chris starts off the discussion by talking about the emergence of crypto tokens, and a look at how videogames and gamers were early to the idea of community engagement and digital assets, and how that model is beginning to spread outward.

      Show Notes

      • The growth of NFTs from gaming tokens [1:20] and how they might be used in the real world [6:31]
      • Issues of portability and security [11:56] and the fractionalization of NFTs [16:16]
      • How NFTs and fungible tokens may affect music creators, including relating with fans and ownership rights [19:46]
      • Impact of NFTs on writers and how digital tokens could be used to fund new ventures [23:49]
      • Social tokens and NFTs as part of the same economy [26:27]
      • The role of big tech platforms [28:16]

      Transcript

      Expansion of NFTs and gaming tokens

      Chris: We’ve all…all three of us and our friends have talked about this stuff for years. I think we’re starting, this year, to see what you can kind of call app layer mainstream kind of crypto token things happening.

      And so, that, of course, is a lot of people have heard about or, you know, originally, I think it starts with NFTs, so non-fungible tokens. But Kevin, you’re working on, is sort of the fungible token counterpart to that, which are tokens that would be associated with communities on the internet. I kind of think of it as analogous to how modern video gaming works, where you have, a game like Fortnite, and the most progressive games. The game themselves are free, but you have in-game currency. In the case of Fortnite it’s called V-Bucks. And then you use that currency for various things, including for buying digital goods, like, in the case of Fortnite skins, and emotes, and things, which, you know, in the web world, the V-Bucks would correspond to social tokens, and the virtual goods to NFTs, right?

      And so I think what I believe is sort of happening now is that video games, which are the most advanced in thinking about how to engage people in social software, and in a way that both goes viral and spreads on the internet, but also makes them money. Those ideas that have been developed over the last 10 years in the gaming world are now propagating out to the rest of the internet, in the open internet. And that, of course, is going to have some similarities, including a lot of design overlap, I think, but also differences, in the sense that these crypto blockchain concepts exist on the open internet and not within silos.

      That’s where I feel like we are. And I think the first bit of the kind of sunlight has broken through in the NFTs and people are starting to see it, but there’s a whole bunch more hopefully coming in the near future.

      Kevin, maybe you could describe how you think that might evolve over the next year or two.

      Kevin: The gaming world is a little bit unique because we created these online communities that had a deeply integrated, set of social interactions and communication tools.

      And so, you know, before there was social media, there were these games that these nerds like me kind of hung out in, and we developed our friends and communities, and, played these games. And we cared about what our mounts looked like, we cared about what our skin looked like, and how we appeared to the rest of the community that we developed relationships with. And, you know, today, you don’t need a game for that. You don’t need a World of Warcraft, or EverQuest, or something like that.

      Now you have Twitter, you’ve got Facebook, you’ve got Snapchat, you’ve got TikTok. The game is not just in the game anymore, it’s happening across all of social media. It’s happening across forums, and Reddits. I mean, it’s happening everywhere.

      And if we could build at the blockchain layer, how do we then take that and propagate it all across the internet, and not just have these things be in games?

      Chris: Yeah. I mean, Jesse, you have a long background in music. So, like, the video game industry, I think, is something on the order of $150 billion, with a B, per year in revenue. And I believe the music industry is something more like 20, and for the most part it’s not really grown with the internet. And why isn’t a musician, with a community on YouTube, or Twitch, or some other place, just kind of an MMO. Instead of shooting other cartoon characters, you’re listening to music and talking to people. You know, and why can’t they take advantage of all the same monetization and engagement technologies that the gaming world has developed? Certainly, there’s no lack of passion for music, you know. People are just as passionate in those environments, arguably, more so than in games.

      Jesse: Yeah, well the sort of recorded music industry hasn’t grown all that much in the internet age, one thing that’s interesting is, like, the live music industry has grown. People are definitely engaged with musicians.

      Chris: Well, but what they’re doing is, that’s the virtual…they’re monetizing the complement, right? So they’re using the internet as the free part, and the offline is the premium, right? To analogize, to, like, productivity software and freemium models. But there’s no reason they couldn’t have a scarce resource on the digital side as well, right?

      Jesse: And I think that’s what’s been missing today, is that, you know, music became free with the advent of mp3s and piracy. And then Spotify, sort of, you know, won by making accessibility super convenient. But with that, you also sort of demolish the value in the sort of scarce creative work that artists are producing. And I think NFTs have reintroduced, you know, the concept of scarcity to the digital realm, and sort of given fans a new way to patronize creators, express their support, route financial value to things that they want to see in the world. And from there, there’s all kinds of interesting things you can do with them, and communities can build around them, and start to get more into social tokens and the like.

      So, one analogy I think is useful when thinking about social tokens and music is, you know, artists had fan clubs, right? And if you bought a membership into a fan club, you got maybe access to the artist’s meet-and-greet, you know, backstage or something like that. But, you know, we haven’t had a digital equivalent of that today. And I think social tokens might be it. You’re becoming a member of an artist or creator’s sort of community by owning their token. And that probably gets you access to all kinds of new cool experiences.

      In crypto world, a lot of people talk about how early they bought Bitcoin, right? It’s sort of a signal of, like, you know, how O.G. you are or how deep you are in the space. And I think that same behavior definitely exists in, you know, certainly with indie music fans being early to a band or whatever. Now you can prove it and profit from it, which is cool.

      Real-world applications

      Chris: So, Kevin, what does this mean in practice? Can you just kind of walk through what the user interaction is? When will people encounter this? And what will their experience be?

      Kevin: Yeah. I think there’s a few different dimensions So one is how easy is it for the average fan, not the crypto audience but the fan, to figure out how to earn, or buy, or trade either an NFT or a social token. And there’s a lot of different approaches there. There’s very crypto-native type of approaches, where everything has to be on chain. It’s got to be held in a non-custodial wallet, etc., to be considered real. And on the other side of it, what we’re trying to experiment with is how do we make that onramp experience for the average fan pretty simple. Vertically integrate as many different things as possible, have the first time crypto experience for the average fan be something that they would expect from another internet service. Some other approaches like what Zora and others have expressed as kind of putting up a big enough barrier so that the fans will have to figure it out.

      So there’s all sorts of different approaches, and there’s no right or wrong answer. And different young musicians or celebrities will figure out what’s best for their fan community and do that.

      The second dimension is, once you own this thing, how do you actually use it? So it’s great if I have an NFT in my wallet. Okay, maybe I can show people a link to my MetaMask wallet, and people can look at the NFTs that I have in there. But how do I actually show this thing to the rest of the community that cares about it.

      And so there’s a lot of work in terms of just status and reputation, and being able to show off different things. But I think, even more importantly, is how do you actually potentially use these NFTs. There is no doubt in my mind that five years from now, maybe even sooner, your backstage pass will literally be an NFT. Somebody will stop you at the velvet rope, they’ll scan your QR code that shows that you indeed own the NFT.

      Chris: An NFT, will it just be like  a digital equivalent of a backstage pass? I find one of the interesting things about NFTs is that it can be multipurpose. Like, it could be both a beautiful picture and a backstage pass, and an investment opportunity, right?

      Kevin: There’s nothing that says that an NFT can only have one use case, right? Certainly, you’re talking about a financial or economic dimension of the NFT having value of whatever the community or the fan gives it, or what the next highest fan would pay for it. But then you can use it as a backstage pass after the event, right? It’s not like the NFT disappears. I mean, you could certainly configure it that way, if you’re a musician, and say, “Hey, once you use the NFT, it burns.” You could certainly do that. But in this particular case of a backstage pass, what probably makes more sense is that you still own it as a fan. And a year from now, five years from now, it’s proof that I went to this event and I was a fan from five years ago when the band was still undiscovered, or whatever it was. So the NFT could be a collectible.

      A lot of what’s happened in the gaming realm, for example, is creating sets and creating different ways that you can compose different items together. So in the MMO world, one of the primary mechanics that have evolved is taking, you’ve got to get this leather strap, you’ve got to get this gem, you’ve got to get this catalyst, and you’ve got to go get this ticket. And you put all of them together and it gives you this new thing, right? And so what happens if you own a track from the musician, what happens if you then combine that with one of the backstage passes that show you’ve been to an event? And then you combine that with something else that shows that you bought a vinyl or equity.

      So we’re going to see all sorts of different ways you could create NFTs. You can use them, you can then as a creator say, hey, if you go collect a bunch of these other things, you can then forge a new type of thing that you only get by being a true fan of mine.

      Portability and security issues

      Jesse: I think the key thing that, Chris and Kevin, you’re both touching on is that these assets are programmable, right? And you can sort of compose them into all kinds of new use cases, and they’re also portable, and that’s because you own them in the same way you own Bitcoin. It’s yours, you can choose to park it somewhere like Coinbase or you can take it with you to another platform. And so, because they’re both programmable and portable, you can take your assets and bring them into all kinds of new experiences that developers build, that give them different utility. Like, it can be a fan club backstage access pass, but a third party developer can add some additional functionality to an NFT or social token that makes it useful in another context for a different purpose.

      Chris: I think when people start to really get a tangible feel for it, it will make a big difference. So, right now, if you’re buying a piece of art on Foundation or something, or you’re buying a basketball moment on Top Shot, you basically can use it in that context. But because they’re blockchain objects that are portable, third parties will start creating experiences around them. I think you’ll very soon see companies that get funded, that let you do games, and social experiences, and other things with all of these assets.

      And kind of the broader thing is you’re inverting the polarity. The earlier web was built around applications. The next, the web3, will be built around these user-controlled objects, as primary and then the applications come secondary, and serve them.

      Kevin: We’re talking about Flow and NBA Top Shots just absolutely exploding. And then we have OpenSea, and Zora, and a few others on Ethereum, and then there’s emerging some of these Layer 2 NFT, you know, sort of like purpose-built Layer 2s for NFTs and a few other things. And I wonder about this portability and kind of what you guys see as how portability evolves over the next year or two, as this fragmentation of the Layer 1 and Layer 2, things fragment more and more.

      Jesse: So, for social tokens, I think there’s going to be a lot more interoperability because they’re fungible tokens, they’re sort of easier to port around, and it’s okay that they sort of fragment across the universe of various blockchains. NFTs, I struggle a little bit more to reason about because one thing that makes an NFT valuable is the fact that it’s unique, it’s scarce, and therefore its provenance is an important attribute that people look at. And so, I do wonder if there’ll be more of a sort of power-law winner to the place where you want to originate an NFT. It may not be the case that all NFTs originate on this sort of canonical…

      Chris: But couldn’t you have trustless bridges across blockchains that preserve provenance?

      Jesse: Yeah. And I think that’s ultimately the solution. So, I believe in a world where literally every piece of media enters its existence as an NFT. Like, every photo you take on your iPhone, you know, every game asset is created as an NFT. And it probably doesn’t make sense to put all those very, very long tail of media assets on something like Ethereum, which is very expensive because it has a lot of security. Like, you probably put that on a side chain. But then I think as these assets take on social value and start to command more market value, they might migrate to the chain that offers the highest security, right? So, if you have a multimillion-dollar LeBron, you might not want that riding on some side chain, you might want that on Flow itself. And similarly, a photo that starts as inconsequential but becomes very important will maybe migrate to Ethereum for security.

      Chris: I guess the way I think of it is you’d have different blockchains with different tradeoffs. So, right now, Ethereum, of the non-Bitcoin programmable blockchains, is clearly the highest security blockchain. But you pay for it. You pay to do stuff per transaction for gas fees. So you could imagine a world where the actual activity is happening on Rally, or Flow, or something else. But then as it appreciates, you put it in the “vault” on Ethereum.

      I’d also say, I think that a lot of people will frame this as either or, Ethereum versus Flow. If you look at every computing resource in history — so, internet bandwidth, you know, PC, CPU power, just go through them all — demand outstripped supply by 10x. And, right now, you could imagine a world where tomorrow, you snap your fingers, Ethereum has sharding, proof of stake, all the good stuff, Optimism launches, you know, all the other Layer 2 launches. And then application developers would come up with some new clever stuff, and you’d very quickly be back up to not $10 gas fees, but $5 gas fees.

      I mean, look, you just pay all the above. You pay inherent overhead for a blockchain, right? The game theoretic consensus mechanism just makes it slower than in a traditional centralized system. I think the specifics, as you raised, Jesse, like, you know, how do you preserve provenance across chain is really interesting. I think there’s a lot of entrepreneurial opportunity for abstraction layer. So, like, a stripe for minting NFTs seems like a no brainer idea. Like, stripe for NFTs, whatever you want to…for minting and read write, and it abstracts away the underlying blockchain, you know, taking care of the metadata properly. There’s a whole bunch. We need more entrepreneurs because there’s so many great living fruit ideas.

      Jesse: Maybe we could talk about interesting intersections between NFTs and social tokens, So an author minted a blog post as an NFT on Mirror. And they also crowdfunded the creation of that blog post, which is sort of like a longform piece. And they said, “Look, if you want to see me write this investigative piece, back me to do it.” And the way that that crowdfunding happened was through crypto. And what the crowdfunders got was not just the piece, out there on the internet, but also ownership in the piece itself. Meaning, they were able to get fractional ownership of the NFT.

      And that was in the form of a fungible token called the Essay token. And what’s really cool is that Essay token then became a social token for this author, in that he started using it for gated access to a Discord. And he started layering on all kinds of other utility, where if you had this token, you could talk to him about his next piece, for example. So that’s one interesting example where the social token is sort of a derivative of the NFT. And then, as we talked about already, there’s all this sort of, like, additional utility programmed onto it. And I wonder if you’ve seen people doing stuff maybe the other way around or different configurations.

      Fractionalization of NFTs

      Kevin: Yeah. You know, I think it’s really funny. We start with fungible tokens, Bitcoin, Ethereum, etc., and then we create NFTs as a new building block. And then of course, then we take the non-fungible token and we fractionalize it and make the fractions fungible. It’s a funny world.

      I think we come up with some of these weirdnesses, especially here in the United States, because of the regulatory gray areas. And so I think, if you can sort of wave a magic wand and say, hey, like, there’s clear boundaries, and let’s just say we can talk about all the things we want to take about about, utility and so forth, without triggering any other things. I think a lot of people would start with fungible tokens. It’s just a lot easier to think about how do you create a community around something that’s more fungible and can be easily exchanged.

      Chris: I don’t totally agree, Kevin. I think it kind of depends on the community. I think there’s a lot of people. I think we all come from technical financial backgrounds. I just find people like that just kind of have a natural affinity for fungible tokens. I think a lot of the world isn’t like that. I think a lot of the NFT appeal is just that, it’s a picture. It’s a movie. It’s accessible.

      And it’s interesting, and it touches on culture and not just numbers.

      Kevin: I totally agree with you. I’m certainly not suggesting that fungible tokens are greater in any way than non-fungible tokens. I’m specifically talking about once you take a non-fungible token, and you fractionalize it into a fungible mechanism, I think once you start going into that …

      Chris: But the counterargument is Crypto Punks. That’s a great way to organize a community. And so it’s 10,000 punks, they look different. But it gives it this character. I think for a lot of people, that kind of metaphor is a better way to organize 10,000 people than 10,000, you know, indistinguishable, boring … it’s YouTube vs. Excel or something.

      Kevin: Each one of those are unique and different, right? And the characteristics and traits of them really matter. Those Crypto Punks are not fungible in my mind, right? Each of them are unique pieces of art.

      Chris: For sure, they’re not fungible, but they create a community of 10,000 people that feel an affinity for one another. You’re either in the punk community or you’re not. I was pushing back on your point that it’s because of regulatory constraints and things that people tilt towards NFTs. I think there’s a bunch of reasons.

      I think one of the really interesting things that goes on with social tokens is that they’re multiple uses. So, you could want a token for your favorite band to get backstage access. You could want it to be able to buy NFTs, and make donations, and participate in the economic life of the community. Or completely different motive, you could want it because you see yourself as a modern day A&R person who is going to predict the next band and make money off it. And it’s the fact that the very same token has those multiple purposes is very important.

      Because then you have the possibility for the A&R folks, the people that find these early bands, to come in, they kind of bid up the price. That, in turn, in your model, funds the actual musician to make music, right? Which in turn feeds the fandom. So the investment kind of activity, and the fan activity, and the creator activity, have this really nice kind of triangular feedback loop. And it’s the very fact that they’re the same thing as opposed to the old school model where you have sort of the investor comes in, gives money to the musician, buys up copyrights, then sells a different thing to the fans, right? The fact that there’s the same kind of… You see what I’m saying?

      Kevin: That’s right.

      Chris: To me, that’s a very big difference from the old world, and a very powerful difference.

      Digital tokens and music creators

      Kevin: Yeah, We’ve always talked about the internet as kind of this disintermediating sort of mechanism. And I think crypto does it even more at the economic layer of things. And I think what we’re starting to see, we’ll stay on musicians for a little while, like, starting with Chance The Rapper, is probably like the most prominent example of an artist that just wants to own all of his own rights. And more and more are realizing that they get their power from their fans. And that the more that they’re just wholly focused on serving their fans, the more successful they become. And this weird layer of rights ownership, and labels, and publishers, that then distort that versus just how you get your fans to support you to create the music that you love.

      Chris: And then when the fans have skin in the game, the fans become evangelists. And instead of doing all these old school things, advertisements, and other sorts of things, promotions, and all the things they would do in the old days to promote various kinds of creations, now the fans become the promoters, right?

      Kevin: That’s right.

      Chris: Because they have skin in the game. And that’s one of the remarkable things about crypto. Tokens, over a trillion dollars in value, lots of successful companies, you know, exchanges and such, and no one ever spends money on paid marketing because it’s all just done through kind of skin in the game peer to peer marketing, right? So, now musicians can tap into that, musicians, creators can tap into that energy.

      Jesse: Yeah, it means that fans become part of the creative process to a degree. Not only do they have a willingness to pay for the creative work, but they’re also essentially investors in the work itself. And artists or creators that lean into that will find a whole new way to create stuff. Because they can do it as a community. And not just a community that’s communicating with one another, but a community that’s actually pooling resources with one another to achieve things together. So, it sort of blurs the line between creator and audience. So I think that’s a big opportunity that we haven’t seen a whole lot of yet, but is coming.

      Kevin: Yeah, I think 3LAU is sort of experimenting with this, where you sort of sold the first track on his next album as something that would give you creative direction on that track as well as original ownership of that track.

      The best thing about working with creatives, whether it’s a musician, or a visual artist, or a gaming streamer or entertainer, is that, these are naturally creative people that once they get their heads wrapped around how a tool or a cryptosystem works, I think we’re just seeing the very, very tip of the iceberg in terms of the use cases because the technical challenges there will get solved, the friction will get removed more and more, there’ll be a lot of different approaches to this. But I think the most exciting thing is getting these tools into the hands of creatives, that then try all these new ways to create that alignment and community with their fans, and disintermediate the folks that haven’t been aligned with serving that community.

      I think the coolest thing that’s happening, is that we’re creating kind of an integrated way that creators can very simply create all different types of NFTs and denominate it in their social token. And, you know, we just think that those are such natural parings, right? So, you know, today, with just social tokens, we see our artists do things, like, you have to hold X number of our tokens to get access to the trove of music that’s here, or to get access to an AMA event or a virtual concert that we’re putting on, or virtual hangout that we see some of our artists are doing.

      And when NFTs come out, it’ll just be much more simple to say, okay, you hold X number of tokens, you are part of the fan club, but to get access to this event, here’s the NFT that represents that event access.

      And then so there’s a lot more granular things you can do. We’re starting to see artists experiment with things like creating physical representations and then digital representations, and then linking the two things together through their NFTs and social tokens. We’re working with an artist that sells sneakers and other physical merchandise. And then they’re creating NFTs of those designs and owning the NFT. They are now starting to say, well, if you own the NFT, you could trade that NFT in for the physical. We’re starting to see more experimentations with how does the physical and the digital sort of coexist.

      And the beauty of all of this is that when you denominate it in the social token, there’s so many other different economic activities that can happen.

      NFTs based on writing

      Chris: We’ve talked a lot about music and video games. What about other… Jesse, you mentioned writing.

      Jesse: Yeah. The writing one is interesting because it sort of illustrates how this expands in all directions and eventually will touch all creative services. So, if you think about how big ideas come into the world, very often they come into the world through blog posts or writing. Think about, Elon Musk’s secret master plan for Tesla, right? Like, that’s sort of a canonical blog post. And certainly, someone might want to own that blog post as sort of a representation of all the value that Tesla is creating in the world. And it’s an investment in Tesla’s success that’s separate from Tesla’s stock price, potentially, right?

      Chris: Owning that blog post is sort of like owning a kind of signed copy of the blog post, so to speak. It’s the cryptographically guaranteed one. Maybe kind of analogous, you know, if, I don’t know, Thomas Edison had a…I don’t know if they have it, but his original notebook or something. You know, that type of artifact.

      Jesse: Right, exactly. So the idea here is like owning Elon Musk’s secret master plan as an NFT, could be valuable. Jack’s first tweet as an NFT is valuable. It represents the sort of inception of his huge network. You can imagine, in the future, that the next Elon Musk or the next Jack brings their big idea into the world as a blog post that is an NFT. And, you know, supporters of them as founders, or people who want to see that idea happen in the world, crowdfund to buy that NFT from the author. And potentially, that crowdfunding is actually used to finance the creation of that big idea. So, you know, what starts out looking like, oh, people are just buying essays, could actually disrupt the way that new creative things enter the world and they’re financed.

      Chris: I guess what we don’t know is, will the economics work? I guess, one counter argument would be, such an exceptional blog post could be worth a lot, but the average stuff won’t be. I guess the counter-counter argument is the average person doesn’t usually need that much money. It’s sort of the Substack effect. If you take a writer with a million Twitter followers, who is getting paid X amount per year, and they go on Substack, and they get 1,000 people who are really excited, they can make 10x per year. Even if it’s not all some famous artifact, if it’s enough to fund the writing, that could just be enough to transform that industry and the creative activities there.

      Jesse: Yeah, and I think… I mean, maybe an analogy is, you know, startup funding, right? Like, startups don’t raise many millions of dollars in their first round. They raise a little bit, just enough to hire the team and get going, right?

      Chris: Sort of staged NFT sales.

      Jesse: Yeah, yeah, exactly. Like, I think you just need… I think, to your point, you just need a little bit to get going, right, and to raise money to make the creative work you want to see. And then from there, if you can build a bigger audience around that, you can sort of move up the ladder and raise subsequent rounds of funding. And hopefully, as you perform better in the market, you’re able to double down and reinvest to keep doing that.

      Chris: And Kevin, your model, that would be both NFTs and also a writer can just sell their coins as well, right?

      Kevin: We think of tokens and NFTs all operating together in a singular economy. This is just something that certainly comes from the videogame industry, where you would expect that you go into a video game, just using the Fortnite example of, get the V-Bucks, the V-Bucks allow you to buy all sorts of different things in the game. And when we talk about things like backstage passes, or that essay, or something else, those things are best as an NFT.

      But then what do you do to transact with that NFT? What if you, as an artist, let’s say your first creative work sells for $10,000. Let’s say at that point, you create your social token, and you denominate your NFTs in your social token. Well, now you have a way for people to say, okay, great, that creative’s first work was worth $10,000. Their second work, the audience valued it at $20,000, and their third work was $30,000.

      The social tokens should then capture what the market thinks about the sort of total value of that economic output will look like over time. And so I think there’s really interesting economic forces between how you create your NFTs and what does that represent? And what does your social tokens represent? And how do they all work together in a singular sort of economy that you, as the creator, you control, you own 100%. And I think that’s a really powerful way to both give all forms of fans, and community members, and crypto members, kind of a way to participate in these economies. You may want to, you know, buy that NFT, because you’re a true fan and you just love it. You may want to buy that NFT because you’re an A&R and you want to speculate on what these things could potentially be worth in the future. Or you could just participate in the social token in all sorts of different ways. I think we’re going to be on the forefront of experimenting with how these things are intertwined, and all put into the control of the hands of a creative.

      The role of big tech platforms

      Chris: What do you guys think the role of the large tech platforms will be in this world?

      Kevin: Well, I’m pretty passionate about this. I’ve tried to build businesses in the past, certainly that have been at the mercy of some of these big tech platforms. I look at what Epic is doing right now, and I know, Jesse, you were at Spotify for a while. There are very public emerging battles between big tech and some of these traditionally more application-focused developers. And I love what this does to the world.

      As I was thinking about building a new company or building a new project, and thinking about building that on Ethereum was so liberating. Because I’ve been building on Apple, Google, Facebook platforms for a decade-plus, as a game developer. And there’s a ton of benefits and value that comes with it, but there’s a lot of headache too that comes with building on these other platforms, where the policies are changing all the time, the fees are changing, the rules are unclear. Maybe they the platform ends up competing with you in the case of music or some other categories. So it’s tough.

      This is why I’m so in love with crypto as a builder because building on Ethereum, I don’t need to worry that Ethereum is going to try to go public someday, they’re going to change the way that the rules work or the fee structure works, so that they can meet their numbers for the next quarter or whatever it is. There’s not even a company, there’s not a CEO that runs it. The idea that this thing is a permissionless blockchain that anybody can build on top of was such a game changer for me as a builder.

      And I think our approach to Rally was to do a lot of hard work so that we can make the same promises and commitments to the creatives that we work with. If we work with you to help you build your business and represent your brand, your fan audience, your community, through tokens, both fungible and non-fungible, you own this. You set the rules. You know, we do things at the protocol level to ensure that all people can participate equally and fairly with transparent rules, but we want to make sure that you, as the creative, you truly own this thing.

      Jesse: Yeah, what’s happening in crypto definitely flies in the face of the way big tech platforms work right now. I think another lens to look at it from is just a complete inversion of their business model. And that’s because, like, you know, traditional big social media platforms, they own all the content that users post on it. And that’s because somewhere along the way, in the terms of service, users agreed to upload content to the platform for the platform to monetize it as they see fit. So, to be clear, I’m not talking about traditional copyright, like, the creator still retains that. But you are transferring some rights to the platform to monetize content that you upload, however they want to do it.

      And with NFTs and with social tokens, the amazing new thing enabled by web3 and crypto, is that creators can just monetize directly. When you create an NFT, you’re sort of like uploading your work to the blockchain directly, and then developers can build on top of that content. In other words, the blockchain becomes this sort of universal library of media that any developer can build a social feed on top of or content feed on top of. But the creator retains ownership of their work and thus can monetize it without a third party taking a large cut. And so, these platforms traditionally have relied on being able to monetize creators’ work on their terms, and now creators get to set the terms and monetize directly.

      Kevin: And if you start with that kernel of the creators create the content and then developers build interesting metadata and usage around that content, that then becomes the social graph itself.

      So think about how, for big tech platforms that rely on marketing and advertising, yes, they create a simple platform for users to share, and create content, and build an audience and following, but it’s really a lot of the metadata around that content, who’s following you, who’s liked what in the past? I think what’s going to invert now, is, once you start with the content being on chain, who owns that content? Who’s owned it in the past? What are all the other metadata that’s associated with that? If you can see all of that, and that exists at the public blockchain layer, you then take it away from being this treasure trove that an advertising-based company can uniquely have as their advantage. And you open it up to the whole world anybody can look at that data. Anybody can look at that social graph and interest data, and then figure out how to build unique new applications and services on it.

      Jesse: Yeah, right now, on social media, everything is sort of in 2D, right? Like, you have an image and it’s just a rectangle on your screen. And then you have some metadata associated with it. But if I copy-paste that image and put it somewhere else, it loses all its connection to the creator, its history, what it’s about. And now all that metadata can live on chain, any developer can access it. So, as a result, you know, this image goes from being a two dimensional box, to taking on some Z-access, where you can peruse through its entire history online, and put all the context on display in new areas where that image is shared.

      Through that same channel that information is being surfaced, value can also flow. There’s this cool thing you can do with NFTs where you can impose royalties that flow back to the creator every time the asset changes hands. So that’s an example of, through the same channel, that information on who owned this thing in the past, well, value can also flow through that same channel.

      Chris: Awesome. Thank you, both Kevin and Jesse. Great talking to you.

      Kevin: Thanks for having us.

      Jesse: Yeah, thanks.

      • Kevin Chou

      • Jesse Walden is the founder and managing partner of Variant Fund, investing in crypto networks and founders. Previously, he was a partner at a16z Crypto and cofounder of Mediachain (acquired by Spotify).

      • Chris Dixon is a general partner at a16z, where he leads the crypto/ web3 funds. Previously, Chris was cofounder & CEO of startups SiteAdvisor and Hunch (acquired by eBay); and an early blogger at cdixon.org.

      All about NFTs

      Jesse Walden, Linda Xie, and Sonal Chokshi

      This episode is all about NFTs. It seems like nothing has caught on and spread into mainstream interest like NFTs, where one hears everything from “I’ve never seen anything like this before” to “is this like ICOs all over again” to “it’s just a jpg I don’t get it” to “but what about the energy use!”

      So, in this special deep-dive episode from the a16z Podcast network, we break down everything you need or want to know about NFTs — while cutting through the noise for what’s hype/ what’s real, as well as where are on the long arc (and sometimes seemingly sudden tipping point!) of innovation (apparently, Google trends data showed that interest in NFTs recently surpassed interest in cryptocurrency). Editor in chief Sonal Chokshi interviews friends of a16z crypto Linda Xie, co-founder of Scalar Capital and former Product Manager at Coinbase; and Jesse Walden, founder at Variant Fund and former co-founder of Mediachain Labs (which was acquired by Spotify, where he was then an R&D lead).

      This episode is posted on both the a16z Podcast show and 16 Minutes as one of our “2-3x explainer episodes” of topics that keep coming up over and over again in the news (past such episodes have covered everything from Section 230 and Tiktok to GPT-3 and the opioid crisis).

      Show Notes

      • What NFTs (non-fungible tokens) are — as well as the properties of crypto that enable them, just to set some big-picture context [2:07]
      • What forms they take, and what is and ISN’T an NFT — including where “social tokens” and the creator economy do and don’t come in [11:18]
      • Common myths and misconceptions — from ‘just a jpg’ to the frequent question of energy use & NFTs [18:23]
      • How they work — as well as the broader ecosystem around NFTs [29:35]
      • Discussion of different ecosystem players [35:08], including DAOs [44:02]
      • Various applications, now and next — touching briefly on how to think about NFTs, whether you’re an artist/creator, developer, or institution [49:32]

      Transcript

      Sonal: Hi, everyone. Welcome to the a16z Podcast network. I’m Sonal, and this episode is ALL about NFTs. And, as with our other special deep-dives, we cover everything you need or want to know about NFTs, while cutting through for what’s hype, what’s real, as well as where are on the long arc (and sometimes seemingly sudden tipping point!) of innovation.

      We start for the first 10 minutes by discussing what NFTs are and how crypto enables them, just to set some big-picture context. The next 10 minutes, we cover what forms they take and what is and ISN’T an NFT — including where “social tokens” and the creator economy do and don’t come in. Then for the next 10 minutes we cover common myths and misconceptions, from “just a jpg” to later going into the frequent question around energy use and NFTs. But about 30 mins in, we quickly share how they work, as well as the different players in the ecosystem.

      Throughout, we of course cover various applications, now and next. And finally we touch briefly on how to think about NFTs. Whether you’re an artist/creator, developer, or institution, this episode is for everyone.

      And, as with past such 2-3x explainer episodes (as I call them), it’s being posted on both the a16z Podcast and 16 Minutes feed, our show where we talk about tech trends in the news, including topics that keep coming up over again [ICYMI: past episodes include explaining Section 230; Tiktok; GPT-3; and the opioid crisis – you can find all of those at a16z.com/16Minutes. We also covered the historic auction at Christies this month, where an NFT by the artist Beeple sold for $69 million; and I mention that only since we reference that event in this episode.]

      Finally, since our podcasts bring you insights directly from the experts, the guests I’m interviewing today are two close friends of a16z crypto: Linda Xie, co-founder of Scalar Capital and former Product Manager at Coinbase; and Jesse Walden, founder at Variant Fund and former co-founder of Mediachain Labs, which was acquired by Spotify, where he was then an R&D lead.

      To be clear: NONE of the following should be taken as investment advice. For more important information please see a16z.com/disclosures.

      Terminology & foundations

      Sonal: So. Let’s just start with a quick set of definitions: What IS an NFT?

      Linda: So, NFT stands for “non-fungible token”, which is just a term used to describe a unique digital asset, whose ownership is tracked on a blockchain.

      This can be a really broad set of assets from: digital goods, like virtual lands and artwork; to a claim on physical assets, like real estate or clothing items.

      Sonal: What I heard you say there is not just digital, because it *can* cover something physical as well, that you can essentially represent as NFTs.

      Linda: Yah. It’s a really really broad space; it’s exciting to see NFT art really take off, but this covers a lot of different industries as well.

      Jesse: So, I like to focus on the digital side of things a little more, and, a metaphor that I would offer as a definition is NFTs are a way to make digital files ownable — instead of a financial asset, you can now own a digital media asset on the internet.

      And that’s why the file metaphor is apt: You can now own a JPEG, own an MP3. And, what you’re essentially doing when you create an NFT, it’s sort of like metaphorically ‘uploading’ that file to the blockchain — such that anyone can track its provenance and attribution.

      Sonal: So, Dixon described this in a recent blog post, very simply put, as: “NFTs are blockchain-based records, that uniquely represent pieces of media” — or in your words, Jesse, a file.

      One more word to focus on is the “fungible” in the non-fungible token, which is that you can represent these items uniquely — I just want to really emphasize ‘cause, again when you think of $1, that’s fungible (or even a single bitcoin arguably is fungible) — but something fungible is interchangeable, replaceable; it doesn’t matter what dollar I have, as long as I have a $1.

      But in this case, something is “non-fungible” means it’s super unique; and we can go into like what that means in a moment — but before we do, let’s talk now about the underlying crypto aspect of NFTs… not the specific crypto protocols, but maybe more broadly, what are the properties of crypto — ‘cause we don’t wanna make this conversation about crypto per se, but about how crypto enables NFTs.

      When you think about the physical world: sometimes it involves a notary; like a third-party bank; it involves someone to (in the art world), like provenance-tracking through certificates… — this ability to own and track a digital file, without a third-party player intermediary, is key.

      Jesse: That’s right, you depend on the bank to maintain the ledger; or the title to a property that you buy, there’s some property registry that the state or the city maintains. So you’re always dependent on a third party to track the attribution of ownership: how the title changes hands, how bank statements get updated.

      And bitcoin changed the game because it enabled this public, decentralized ledger — where no one party is in control — and yet each individual owner of a bitcoin is able to verify their ownership using cryptography. As a result, you don’t have to depend on a single third party to verify ownership.

      Linda: Yeah, middlemen are tracking ownership for people of all these different assets — and they’re taking fees for the service; they’re preventing some people from using the platform — and, what’s really powerful with crypto is you have all these open protocols that you can kind of plug into each other.

      And so, when you have NFTs, you can plug them into decentralized systems and be able to trade these NFTs with anyone in the world, and have that be instantaneous. You can also imagine plugging into using your NFT as collateral; so let’s say you have video game items that’re worth a lot of money, you can actually imagine taking a loan out from them.

      And so NFTs on the blockchain allow anyone to permissionlessly own, issue, trade them.

      Sonal: And the other property of being able to track provenance; which has essentially a built-in secondary market to it — which is this idea that not only do you track the provenance, but you can actually track the financial benefits that accrue as a result of that built-in secondary market. This is particularly true in cases of digital artworks, etc.

      Jesse: Yah, the secondary *resale* of an asset can be programmatically constructed, such that anytime the NFT changes hands, a portion of the resale value goes back to the original creator.

      Sonal: And by when you say “programmatically”, automatically, that is a distinct property of crypto — specifically, smart contracts that you can do that type of programmability of a contract.

      Jesse: Yeah, it can be totally automated, totally transparent — Contrast that with royalties in the music industry, which is like a completely opaque system with many layers of middlemen that are each taking a cut, right.

      It’s a wildly more efficient architecture; that’s uniquely made possible by blockchains and smart contracts. The blockchain’s really good at tracking the history of things — Sonal, if you send me one bitcoin, everyone can see that you have one less and I have one more <Sonal: right> — and the history of that transaction is forever sort of enshrined on the blockchain. The same is true of non-fungible tokens in that, when they’re incepted or “minted”, they’re signed by the creator using their cryptographic keys — which now enables anyone to see okay this file was signed by this creator or this person — that message is constructed in the same way any other cryptocurrency transaction is constructed.

      Thereafter, that NFT lives on the blockchain alongside all other transactions, and everyone can see it. And so if that NFT changes hands, and say Linda buys my NFT, everyone can see I transferred ownership to Linda. And as a result, we start to build this very rich history of the interactions people have with media on the internet. Whereas today, think about an image you see on Instagram: You could screenshot that; you know, crop it; and then paste it on another platform, say like repaste it on Facebook. And as soon as you do that, you break its entire history, its entire provenance — you no longer know who made it, what it’s about, where it originated… And, with this new sort of architecture, we can now sort of have a z-access into the entire history of any piece of media on the internet.

      And: through that channel that information flows, value can also flow.

      Sonal: Concretely for artists, this means an artist today who may have created a work 20 years ago — and that work completely appreciates in value, but the last owner is the only one who benefits from that — if the programmatic arrangement is that that artist continues to get value, they can always get paid on this built-in secondary market. Like, if it later becomes millions of dollars versus $500 for a painting, then you’re getting money back each time it is sold. Which is not possible before.

      Jesse: And I think, important to note, that’s just *one* of the possible arrangements that can exist, when the rules around monetizing creativity can be expressed as code by any developer and any creator on the internet. 

      Linda: The ownership history is really important — the ownership history is something that is really uniquely accessible on the blockchain, because everyone can see, and therefore some items might be more meaningful to certain people. Let’s say “Magic: The Gathering” has a tournament where this deck of cards actually won this tournament, you might want to buy these set of cards because they’re historical, and the winner of these games actually used this deck to play.

      From the art perspective, just imagine your favorite musician or creator owning a piece of art. And, now that ownership is just tracked in the blockchain, that piece of art might become more valuable to you because of who’s owned it in the past.

      We also have a lot of projects that are working on fractionalizing NFT art: So, splitting up these NFTs into multiple pieces; and these individual pieces are also tracked on the blockchain, and you can trade them through decentralized exchanges.

      So, it’s really powerful when you can plug these NFTs into all these different crypto protocols, because in a traditional system, these middlemen aren’t plugging themselves into all these other companies and middlemen. You can kind of freely do whatever you want with these NFTs, which I think is a really big difference.

      Jesse: Yah, I think it’s important to contrast the way NFTs work to the way the traditional web works; so, with social media today, when you share a file or share a piece of media, you upload the file to the platform. And what’s actually happening under the hood is you’re “copy-pasting ownership” of the file to the platform: What I mean is that somewhere along the way you signed the terms of service that allows for the platform to monetize that piece of content as they see fit, and maybe they give you a cut of the revenue, maybe not — but the platform gets to make that call. And, they also get to make the call on how that content is consumed, and there’s not a whole lot of innovation going on there because any developer who plugs in to try to innovate has been shut down in the past.

      Now, contrast that to NFTs — and I’m going to run with this metaphor of uploading a file to the blockchain —

      Sonal: Keep goin’, I love someone who owns a metaphor… <chuckles> <Jesse chuckles> do it! <laughs>

      Jesse: Okay, so, with this metaphor if you’re uploading files to the blockchain, and then those files become NFTs and they behave in the way that other crypto assets behave, that means that they’re permissionlessly accessible to anyone, anywhere with an internet connection. The implications are that any third-party developer can then innovate on the way that media is consumed: Like how the audience sees it, how people can interact with it or program it.

      So, one way to think about what’s happening with NFTs is we’re building this universal,  open media library — on top of which any developer can build the next Spotify, or build the next Instagram, or build the next Facebook — and when there’s a lot more competition, there’s gonna be a lot of benefit to consumers… and likely to creators as well — because as Linda mentioned, all of this can happen without the traditional middlemen taking a cut of the value that’s flowing between the creators and consumers.

      Sonal: That’s great.

      What an NFT is and isn’t

      Let’s get a little specific, though — let’s actually talk about the forms NFTs are taking, specifically. I think this is a great place to help tease apart what’s hype/ what’s real, as is the premise of the show. And so far, I’m actually having a hard time — and I’m someone who’s been covering this space: I mean bitcoin since very early on, Ethereum since very early on, NFTs since very early on — and I am honestly confused myself! So, maybe you guys can just help break it down.

      Just to quickly recap, Jesse, you’re saying any media file; Linda’s saying any good, digital or physical — that leaves pretty much anything… So specific examples include things like:

      • art;
      • it can be in games;
      • in music — there are audio NFTs (this has been really interesting to me lately);
      • there are blog posts, like I see people on our friend Denis’s site talking about making NFTs of blog posts;
      • Brian Flynn wrote about token-gated newsletters
      • And another interesting example recently (it was a self-proclaimed first, but likely true): Someone wrote about how they created the “first ever tokenized crowd-funded equity research report”.

      …Can you guys just quickly help tease apart what *is* and *isn’t* an NFT? It seems like everything is!

      Linda: I really think that anything <chuckles> in the world can be an NFT… as in, anyone creates something that is unique, that can be owned.

      The problem with physical goods is that you do have to have someone custody it, so there is a process behind that of having to make sure that you can audit that in the real world: And maybe multiple people own it, and it can’t be moved just by one person; so, there are pieces to that — but otherwise I find it to be an extremely broad category.

      We’ve seen really cool things come about, where we’ve had the token-gated newsletters, which you mentioned — basically needing to have a certain number of tokens in order to access this newsletter. And people are doing like token-commissioned permission chats where these tokens are required to enter the chatroom and start talking so, you know everyone has like a level of skin on the game. You have to own like a certain amount of tokens in order to enter these groups: it proves that you have some sort of ownership into this community that can be adjusted over time. (The idea is that even one day that there could be DAOs formed where token holders can vote on how many tokens are required to enter this newsletter or chat group or something.)

      And that piece, it’s kind of tangential to NFTs — I don’t necessarily think that social tokens themselves are NFTs, because sometimes people are creating these tokens that they’ve minted like a million of them; but if the creator of this group is issuing individual badges or unique items within that, then that can be an NFT.

      Sonal: Can you say more on what you mean by “social tokens”?

      Linda: Yeah; social tokens are just a really broad category of tokens that are issued by individuals or communities. So sometimes it’s- other terms are used like personal token, community token, creator token — but social token’s kind of the term that encompasses all of it.

      And there’s just a bunch of different experiments happening in the space: So we’ve seen people tokenize their time; so one of these tokens equals one hour of their time, and that becomes freely traded. We’ve also seen someone like R.A.C (he’s a Grammy Award-winning recording artist) tokenize his social token, and his token holders get access to this private Discord group, and then they receive like all these additional benefits… and he retroactively distributed to his supporters — so this is a way that creators can interact with, and reward, their early supporters.

      So it’s a really broad category and it can really be anything associated with an individual or community.

      Sonal: And how is social tokens similar and adjacent to NFTs — and then when is it *not* NFTs? Can you help distinguish there, just to help the understanding of what is / isn’t an NFT.

      Linda: Yah, sometimes social tokens can be NFTs in that a creator is issuing some unique piece of artwork directly to their fans — but in a lot of cases, social tokens can also just be fungible tokens. So like R.A.C. token, they’re all fungible so you can basically hold like a certain level of them, and that can always be traded and bought back and it doesn’t really matter which R.A.C. token you’re purchasing. So those are kind of more adjacent.

      And I think the reason why social tokens and NFTs get lumped together a lot of times is just because it enables the creator economy, it enables creators to engage with their fans directly… and so, these are often tied pretty closely together.

      Sonal: Right, but basically the bottom-line is — if it is fungible, it’s not an NFT; and if of course if it’s non-fungible (hence non-fungible token), then it is an NFT.

      Linda: Yah. 

      Jesse: I think the line between fungible and non-fungible tokens is blurry for a reason, and that’s because the interplay between the two is enormous:

      • You can take a non-fungible token, and turn it into fungible tokens by fractionalizing it;
      • And then you can make those fungible tokens — which represent a piece of the original NFT — you can make those into a social token.

      So there’s this token called B.20 which is a fungible token. And it represents a claim on some of Beeple’s NFTs, which an investor bought and essentially fractionalized ownership to. And now that that B.20 token exists, it can be programmed into all sorts of other value: So in addition to owning a piece of a Beeple, you can imagine some third party spinning up say, a Discord server and saying you need a B.20 token to come in here and hang out. That’s an example where it started with a non-fungible token; Beeple created it; then a collector bought Beeple’s non-fungible token; fractionalized it into a fungible ERC-20 token (that’s the B.20 token) — and third-party developers can remix and add new experiences.

      Linda touched on all of this, but the interplay between the two is important to note that you can easily take a non-fungible token and fractionalize it into fungible tokens, that then can become these social or community tokens. And so that’s a fun sort of design space to explore.

      Sonal: I mean we’re talking so far about kind of digital versions of what already happens in the real world, being able to do things in different ways. But if you think about what’s NOT possible right now in the real world, the idea of fractional ownership is super important and goes to the things that crypto can uniquely do. ‘Cause right now, if you want to fractionally own an artwork — I mean, I guess there are some very kluge-y websites where you could go and kind of combine resources with somebody — but guess what <chuckles> you can’t really split a physical artwork! So, it kind of reminds me of that story, I don’t know, it’s like a Bible story about like splitting the baby; like, who’s the real mother? But the point is, you can enable fractional ownership.

      I still am stunned at this idea with Top Shot, that you can essentially buy and own a *moment*, a favorite hit moment in a physical sports game. And by the way Linda, I’ve been meaning to text you about this, but I’ve been addicted to watching CLOY (Crash Landing on You, the K-drama); honestly when I think of all these amazing moments in a show like that, I know this sounds nutty, but what if people could bid and own moments on their favorite TV shows — as if it’s theirs — even if millions of people can watch it! Like, the idea that you can own it, and it’s part of your identity, is so freaking awesome.

      Linda: That’s what I love about crypto, you can have these concepts that you never really would have thought about otherwise — about being to own moments, and media, and people’s lives — imagine like YouTube creators streaming what’s happening to their life, and be able to own really exciting moments of it. I mean there’s so much possibilities there… it’s really exciting to see people become so creative with NFTs and what you can own.

      Common questions

      Sonal: On that note, let’s actually switch to what’s overblown or not, and tackle some common myths and misconceptions — everything from energy to whether it’s a hype cycle. And of course we’ll keep talking about the applications, but let’s first dig more deeply into what it means to own something digital.

      Start with this common phrase: “It’s just a JPEG”. How would you guys address that comment — “like ohmygod someone spent f’ing $69 million for just a JPEG?!” I mean, Jesse, you’re saying it’s a file. Guess what? On the internet, files are pretty worthless, and infinitely replicable.

      Jesse: So I would say that the number of times a file has been reproduced on the internet, is directly correlated to the value of that file’s NFT — meaning the more times a piece of media gets shared online, the more social value it has.

      To make this concrete, it’s helpful to think about a very well-known piece of traditional art, like the Mona Lisa. Where the Mona Lisa has been reproduced probably a zillion times — it’s on every t-shirt, postcard sold at the Louvre; you can see it anywhere on the internet — yet there’s ONE Mona Lisa in the Louvre Museum, and ownership of that piece of artwork is incredibly coveted, incredibly valuable.

      Sonal: I have to admit I’m getting a little tired of the Mona Lisa analogy, but it is a useful one <chuckles> — the key point and what I really heard you say, which is so counterintuitive, is that something is more valuable the more that it is replicated…

      And in fact, it just makes me think of how in general, as the world of the web became more about abundance versus scarcity — you know, the long tail is that you can have infinite choice on the web — it is really fascinating how people have been, when they went past the limits of the Blockbuster shelves in the physical-goods world, it’s interesting that Netflix would do things like binge seasons and drops, that kind of created this digital scarcity — like a limited-edition effect; like this thing is going to be on for three months, and then it disappears.

      So it’s another analogy of this interesting relationship between something not being necessarily rare or scarce — and in fact, even infinitely replicable in the case of files and JPEGs — but you can enforce this digital owning, or even a piece of it (if you want to go into the fractional ownership bit), and that is incredibly unique.

      Jesse: That’s the idea — is you’re not owning to try to be the ONLY person who can access a given JPEG or piece of media — rather you want to own the piece of media that everyone else sees.

      Memes, you know, travel the internet at a rapid clip; they get shared infinitely — you can now own a piece of internet history, or the most viral meme. And I think very soon we’ll see that these owners are credited, socially, on platforms where that work is distributed.

      Linda: We’re seeing that play out with CryptoPunks, which is just one of the earliest NFT projects. And we saw two CryptoPunks sold for $7.5 million each — one of the sellers being Dylan Field, who’s the CEO and co-founder Figma — and what’s really interesting about this is, yes, anyone could just copy this image…

      Sonal: …He, himself — sorry to interrupt you — he, himself, could copy his image, because he had it as his profile photo for a while, and he had to like (he didn’t have to, but he chose to) take it off his Twitter profile. Which I thought was so funny. But keep going.

      Linda: Yeah, exactly. So anyone can copy this; but if you actually look at the CryptoPunk NFT itself, you’ll see who owned this.

      And the fact that somebody had owned this so early on… it’s almost become this status symbol where people want to demonstrate that they can prove their ownership really early on in this space. And so that ownership history also really matters: Being able to discover an artist or creator really early on, and being like one of the first supporters, and having that tracked on the blockchain. Future times when that is sold, you can prove that you were this early adopter, and that is very valuable to people.

      Sonal: There’s a great analogy I heard, to extend what you said even further — but basically, if you think of the placard next to an artwork like at MoMA or some famous museum — it tells a description of what the art is, the materials, the date; the artist, if it’s not someone anonymous — and then it says at the very bottom, you know, sometimes in very small letters, who owns it, or it’s lent by someone on this collection so-and-so. And not everyone pays attention to that, because most people just care about the art, and then some people actually care about who owns it.

      But now, you’re doing that exact same thing but on the internet, where the whole world can see it — not just like going physically to MoMA and seeing it. Or even to use Jesse’s analogy, Mona Lisa and seeing it in the Louvre; it’s like not only can you have that kind of insider notion of the ownership, but you can make it more outsider facing by letting everyone see it. That is pretty powerful.

      Jesse: Yah, and developers are gonna lean into that. Because all that data about who owns it, where it came from, what its history online is — that’s just an API call away. And, right now, again it’s very difficult for developers to find all that information on web-2 platforms because the history of media is not tracked architecturally in the way that blockchains track it.

      So it’s like a 100 times easier for developers to surface that information — and that’s why I think you’ll see that little placard in the museum — you’ll see the digital equivalent of that on all social surfaces in the near future.

      Sonal: Did you guys see Matt Levine on Twitter, he said this line that “NFTs are a new form of tradable ostentation rather than a new form of tradable ownership.” Did you guys have any reactions to that?

      Jesse: I mean ostentation is one of the reasons people might collect NFTs… but it’s not just the speculative value, or being ostentatious about being the owner — increasingly over time, I think owners of NFTs will start to realize more and more sort of compounding utility as developers build new spaces for you to bring your digital property. For example today, you can buy a piece of digital art as an NFT, and you can bring it into a virtual world (like Cryptovoxels or Decentraland) and display it there. And that’s a very early example of a third-party developer who has nothing to do with the creator of that NFT, being able to build a new virtual experience — that’s just the tip of the iceberg; there’s, you know any third-party developer can then build on top of Cryptovoxels because it, too, is open-source and permissionless.

      And so what you start to see is this sort of Lego-block approach to building new experiences, where developers can build more with less, and the innovation compounds much more quickly. And so, that statement undervalues the possible utilitarian nature of being an owner.

      Linda: Yah, I don’t really agree with that statement, ‘cause I think it dismisses a lot of the use cases. And if we even talk about the more traditional stuff — like NFTs representing tickets or financial assets or real estate — these are just more efficient ways of transferring, and without having as many paperwork and middlemen involved.

      And so this is a net benefit to society, versus people trying to display their wealth.

      Sonal: I’m so glad I asked you guys because again the premise of this show IS to tease apart what hype/ what’s real, and it is interesting that someone whose work I deeply respect… has an interesting observation — and to hear you guys push back on that.

      What do you make of the comparisons that people are making to the ICO boom, what would your response be to that? ICO boom being “initial coin offerings” — playing off the term for IPO, obviously initial public offering — but in that case, it was more risky people argued, because an ICO was before the thing even existed. Like at least in an IPO, the company exists. By the way: Our friend Nick Tomaino made a simple observation that an NFT is a concrete product, a digital good, not a promise about the future, which I thought was a good argument. <Jesse: Yeah. I like that.> Any thoughts on what the hype/ what’s real on, “oh no, we’re in another ICO boom; it’s like tokens all over again”?

      Linda: It just reminds me a lot of 2017 when a lot of people came into this space — and the word ICO was thrown out a lot, and there was definitely a lot of hype around that.

      But, through that process, a lot of really incredible projects came out of crypto. And, a lot of people who joined the space ended up staying because they saw that it was a lot more than just-get-rich quick; and there were communities, and really unique ways of creating value in this world.

      So, yes, there are a lot of people that have come into this space, wanna just make some money off of it — but there are a LOT of really creative people that have joined the space and are going to stick around, and experiment with what’s happening, and really build some things that we have just never seen before.

      And it is nice that this time, it’s artists who have just worked so hard and are getting rewarded for this type of work. Someone like Beeple, who has been working on this craft for so many years, this is something that people are valuing.

      Sonal: I mean, on Beeple specifically his work was digital from the beginning anyway, but he’s essentially creating — and this goes back to the definition, this kind of one-of-a-kindness — because it is an NFT, it is unique and trackable that way.

      I do love that… but there’s no question there is like a hype cycle going on, we’re at that moment in whatever the Gartner curve, or whatever the framework you wanna use; there’s always a trough of disillusionment phase… I guess I just need a little bit more to understand, okay yes, this is very exciting — but right now, in this moment in time, how do we assess that it’s working / or not working?

      Jesse: I think the question you’re getting at is what is valuable, which NFTs are valuable?

      And the answer to that question is kind of like answering the question, “what is art?”; and my answer to that question would be whatever the beholder thinks is art.  And similarly, whatever you know, the market thinks is valuable… is a valuable NFT. And that’s why I think you’re seeing such wide-ranging experiments in what can be transacted as NFTs, from blog posts, you know to digital art.

      And there are a lot of niche groups that want to own you know an item that’s culturally relevant to them, for various reasons — whether it’s for speculation, the idea that they might be able to resell it in the future; or, you know, because they just want you know their name on this sort of digital placard next to the item to say “I supported this creator”, right like I wanted to you know support their work.

      So there are a lot of different reasons people might value NFTs, and there’s a lot of different subcultures and value systems that you know comprise this market, and that’s why it’s sort of expanding in all directions.

      Sonal: So the buzz is not necessarily a bad thing.

      One of the things that came up a lot in the early days of the history of NFTs, starting with CryptoKitties, was the scaling problem, and the fact that Ethereum was not ready for that level of excitement yet — and it pushed a lot of solutions into thinking about scaling. So some of the hype cycle in 2017 actually led to good infrastructure improvements, and the installation of things we needed…

      I mean it definitely makes me think that Mediachain was just a bit too early, actually. I remember… even before you guys founded Mediachain, always having this problem in the creator world of having to track libraries of digital assets, it was very difficult to find out — ‘cause the information was not coupled with the JPEG itself — forget even who owns it and who to pay, like you don’t even know who made it. And this is true of memes, everything, on the internet — which you know IS about remix culture, and extensibility, and-and composing things and combining them.

      Jesse: Yeah, a lot of the ideas that are being realized around NFTs are ideas we were exploring back then. And there were two critical things missing from the ecosystem at that time: One was the ability to easily create a token — that’s uniquely enabled by a smart contract, and smart contract platforms like Ethereum, and Flow, and others. The other thing that was missing was markets for these digital assets: So, we now live in a world where roughly 10% of Americans own cryptocurrency. And so the idea that digital assets have value is sort of a prerequisite for digital MEDIA assets, like NFTs, having value.

      To Linda’s point earlier, the 2017 market was a prereq for the NFT market today. So, the markets had to come first; markets drive — you know, they’re volatile, and they drive these speculative frenzies — but they also drive infrastructure, and mental models that stick.

      How NFTs work

      Sonal: That’s a perfect segue to the next thing I wanted to talk to you guys about, which is the broader taxonomy of the players and the ecosystem that’s already emerging around all this.

      And before we do, let’s quickly talk about how they work, to help make it concrete — like step 1-2-3 to minting an NFT, trading it, doing whatever?

      Linda: Okay; step one would just be deciding to put your work as a representation on the blockchain. And so “minting” involves really interacting with the smart contract, and submitting that — there are different marketplaces that try to make that really easy for you to do it. And so some of them will have a button that you’ll click to mint this process, you can select different attributes of like what’s the name of this piece of artwork, what’s kind of the royalties involved if there are secondary sales, like how much do you want involved?

      So, a lot of these will make it super easy for you to go through that process. I think the hardest part actually, is getting set up with a wallet, and onboarding yourself into accepting cryptocurrency and that piece — but there are marketplaces now that make it accessible.

      And some marketplaces will also maybe have you go through some onboarding process — and so they might have some due diligence on the artist; and making sure this is um real artwork and not copied by some other artist; and making sure it’s really high-quality pieces — and there are others that are just like hey, anyone can mint this.

      So, it’s quite a spectrum right now.

      Sonal: That’s great… but, I gotta set up my MetaMask, and like what does that even mean? Can you guys explain the wallet part too as well, ‘cause one theoretically does not necessarily have to actually interact with cryptocurrencies directly — so, if you guys could break down really quickly that bit too.

      Jesse: Sure I can take a stab at that. So, the concept of a crypto wallet boils down to what’s called a public and private key pair — So, basically you have a public address on the blockchain, which is where your assets/ your stuff is associated: So, Sonal has a public address, and you can tweet that out and say, “Hey, I’m Sonal, here’s my public address, and here’s all my stuff.” And your Bitcoin can be at that address; on Ethereum you’d have a different address, and all your stuff on the Ethereum blockchain would be associated with your public key.

      And then there’s the private key — the private key is what unlocks the transfer of assets in your wallet. So you need the private key to unlock stuff at your public address, and that’s the concept of a crypto wallet. MetaMask is the most popular wallet for Ethereum, and it’s a browser extension (you can install it on any popular browser); and, essentially what it’s doing is it’s setting you up with a public address for the Ethereum blockchain, and a private address.

      And what’s critical to note: Is that cryptographic key pair, it doesn’t belong to MetaMask, it belongs to YOU; MetaMask never sees the key pair, they don’t have any of the information, it’s yours. And that comes with a lot of responsibility, because if you lose your private key, you lose all your stuff — it’s kind of like cash in your wallet. And that’s why it’s called a “wallet” (even though it’s a little bit of a misnomer, because you can only do so many things with your physical wallet) but I think the reason that name has stuck, is that your cryptographic wallet behaves like a physical wallet in that if you lose it… all your cash is gone.

      Sonal: Okay, so now continuing the process, so we understand now how the wallets work, the browser extension for the wallet, some platforms can let you like literally create — sorry, mint an NFT — because you can create the artwork in any form you like, or whatever the object is, or digital asset or file.

      Now what happens after you mint it, what’s the next thing that happens? So you can put a name on it, you can specify program terms, you know what kind of royalties, different systems may have different options: Some of them themselves take 10%; others you can program in like as this increases in this much value; I’ve seen people do creative stuff like the artwork reveals itself the further you go along the bonding curve — like they’re doing creative stuff with the art itself kind of interacting, so it’s not just like a static piece of art that just happens to be going through this chain… What happens after the minting?

      Linda: You can do different things, depending on what you want to do with your NFT. So once you mint on these marketplaces, you can just have it listed, and try to share this link with other people, and have them bid on the piece of art. You can set a price; you can have people bid and then accept different bids.

      Or, you could just have this created for yourself: And, let’s say you want to make some worlds in a virtual land, and display your artwork in a virtual art gallery, and just place your artwork there…

      There’s all kinds of different things you can do with it if you think this piece is really valuable. I’ve actually seen people talk about swapping NFTs; so different artists are like swapping with each other. I’ve also seen people put up the art as collateral, and then get out a loan for it.

      So, you can really do whatever you want with this. But the most common, basic thing I’ve seen is just people selling their artwork, and someone purchasing it, and then maybe storing that like on their virtual land, or having it displayed on their profile. And there’s like a social element to it: People talk about it a lot, like hey I just purchased this piece of art, and they’ll talk about it on Twitter. And there’s an app that aggregates purchases from all of these different marketplaces, and displays it almost like an Instagram feed — so you can have like a social element to who’s purchasing what, who owns what, and have people form communities around it.

      Sonal: I’ve also often wondered if there will be like a Pinterest for NFTs, where even if you’re not the owner, you can kind of… “collect” it. Like one of the things that I use Pinterest for is it’s sometimes aspirational stuff I would never, ever buy — and just like having Pinterest boards is a way for me to “collect” it in a different way. Like I wonder if that would even happen, and if people would create, like, fees for doing that as well. I don’t know if you guys have seen that yet, but I wonder about that.

      Linda: I could see something like that happening, and, also just the fractionalization aspect to it — you can cut it up to really really really tiny pieces, and maybe people can own just really tiny elements of this piece as you’re looking through your Pinterest board or something.

      Ecosystem players

      Sonal: So, what you guys just described as all these different steps, there’s a whole ecosystem of players that have now emerged and are continuing to emerge. We’ve already named a few — like sites for showcasing an online collection; displays; like online galleries, we have curated galleries coming up; we have marketplaces; we have other tools for managing details… How would you break down the taxonomy — give me a map, and the terrain.

      Jesse: Yeah so there’s vertical marketplaces, right, where there’s marketplaces for like curated art or for certain types of collectibles. And then there’s horizontal marketplaces, like OpenSea, where that’s more of a search box for all NFTs. And the reason they can do that is all these NFTs live on the blockchain, it’s completely open. So they can query the blockchain and aggregate all of them. And you can find literally, pretty much every NFT on a horizontal marketplace like OpenSea.

      Sonal: Which is great, because not everybody knows how to interface with crypto. This is like the way the web itself evolved.

      Jesse: Right. And then, each of these kinds of marketplaces — both vertical and horizontal — more often than not, allow creators to mint, on the platform. Simple way to think about it: there’s both supply and demand: and you can either get it in the vertical form, which is specialized; or horizontal, which is sort of everything.

      One ofth really interesting phenomenon that’s happening on the demand-side of the market is you’re starting to see these really interesting collector… organizations sprout up: So, crypto makes it really easy to send value — like as easy as sending an email — as a result, people are pooling value in interesting ways in order to participate in this market. One really cool experiment is this thing called Flamingo DAO (DAO stands for decentralized autonomous organization) — the core idea is you can pool resources with anyone with a crypto wallet; send money into this smart contract, that acts as sort of like a bank account, and then that bank account can go and buy NFTs. The group can buy NFTs.

      And so what that kind of looks like is… maybe something like a fund, or you could call it a “gallery” that’s acquiring work. And by being in that collection, the creators’ gaining distribution to the audience of collectors/investors who pooled resources in the first place. So that’s another really interesting phenomenon that’s uniquely crypto enabled. And I think we’re gonna see a lot more of that.

      Sonal: I do too, and I looove that because one of my favorite things in the creator economy in general is the way collectives can emerge, both ephemerally and permanently (I have like a whole tweetstorm about this) — but I think it’s super powerful to think about what happens when you unlock that kind of coordination… Keep going.

      Jesse: Yah. I mentioned vertical and horizontal marketplaces; there’s also adjacent just media platforms — like we touched on Denis’s project Mirror — which is a blogging platform, where anyone can mint their blog posts as an NFT.

      And the question, why would you want to mint sort of a blog post or an essay as an NFT?

      Sonal: Yes, thank you for answering that!

      Jesse: <chuckles> If you’re an investigative journalist for example, there’s not a whole lot of great tools for you to monetize as an independent right now; subscription can be less conducive to long-form journalism. And what’s kinda cool about Mirror is — similarly to the prior idea of people pooling money — it allows for a writer’s audience, to pool resources in the form of a crowdfund: “Hey, I want to see this investigative report written, and here’s the money to do it.”

      And as a participant in that crowdfund, you don’t just become a patron of the creator, or the writer: You become an owner, a fractional owner, of the NFT that they produce when they publish that blog post. And, you can sort of analyze the psychology of one of these backers, but I think it boils down to two things: There’s the idea of *patronage*, right, you’re being a patron of this creator and helping them get the work done; but there’s also this vague notion that, in the future, if this piece becomes very valuable, I’ll be on the “cap table” of the post. Like Elon Musk published his famous blog post, the secret masterplan of Tesla. And just recently, you saw Jack, founder of Twitter, sell his first tweet (ever) as an NFT for millions of dollars.

      So, you can see this idea going a lot further: Where new, big ideas enter the world as blog posts, and people crowdfund those big ideas that they want to see happen in the world, and become part-owners in the blog posts — that becomes the sort of canonical representation of that idea.

      Sonal: I saw Dylan Field post a thread a couple of days ago that I thought was wonderful, talking about some of the extension of ideas around NFTs. He described like “proof of fandom” — and we have lingo in the crypto world of “proof of stake”, “proof of work”; and it was neat to have this idea of “proof of fandom” — it kind of ties back to this idea of monetizing moments as well.

      And in this case you’re talking about ways for creators to have their fans — and one thing we’ve talked a lot about on this podcast; I did a podcast with Kevin Kelly about how you can actually invert the model of payments, where it’s not a creator selling, but buyers and fans monetizing *their* attention — And so the idea of that is super interesting… because you can imagine fans and collectives like buying and owning these things.

      And Dylan even went so far to point out even “community as art” in that example, which I thought was super interesting.

      Linda: So, that’s an area that I’m really excited to see… I haven’t really seen too many people working on this yet, but, the idea of having so many DAO members being able to vote how this artwork looks and kind of have it be collective artwork.

      I was in this DAO called Saint Fame, and we would vote on different parameters of the design of clothing items, and then this DAO would manufacture them and ship them to people that purchased it. And so you had like this group of people deciding what the design looks like, and you can imagine that anyone can join these DAOs — it could even be anonymous people and from all over the world. And so you can collectively create or invest in things together, which is really exciting to me.

      Sonal: Connie Chan (our partner) has often talked about influencer monetization, and she talks a lot about what happens in China with livestreaming and how a lot of fans will ask their audience like “Should I wear this today, and do that?” And some of it can kind of veer on dystopian in some models, but in many ways, it’s also incredibly empowering that you can choose to monetize the things you want to and have models for doing it.

      But right now, it’s the platforms that take all the money. So what’s really interesting about what you just described is that you could essentially do the exact same thing — but in this sense, you’re creating not just artworks, but you’re actually creating collaborative decisions around… a person’s wardrobe, or a fashion line, or however they want to develop products (even physical products) based off of that. Which I think is super fun and interesting too.

      Jesse: Just one last thing to add there, along the lines of proof of fandom, is this idea I’ve been calling “Patronage Plus”: So in Web 2, it’s very easy to become a patron of an artist or creator whose work you admire, by subscribing to them on Substack or paying a subscription on Patreon. And what that essentially does is gives you access to their work, but it also allows being a supporter financially of the work itself.

      NFTs allow you to do the same thing: in some cases, the NFT can give you access to a Discord or a newsletter (and we touched on that) — but the *plus* part of patronage plus is what’s new and uniquely-enabled by digital ownership. And that is the possibility of being able to profit in the future from the resale of that ownership to someone else, maybe as the creator’s profile raises or the demand for their work grows.

      And, I think that “plus” is really key because it’s a very strong incentive to become a patron in the first place. So, patronage plus may end up growing the market way bigger than patronage that we saw in Web 2.

      Sonal: What’s so amazing about that is the golden age of art, many argue — like in the Renaissance era in Italy, Medici family, etc. — people argue that patronage in the first place is what unlocked that. And so what you’re describing is a more democratized form of patronage, and the “plus” is a way to really have this knock-on effect over time — which is really investing and democratizing — in a way that is accessible, to everybody. Because it’s not just the rich Renaissance families that can do the funding of the arts.

      Jesse: Yeah, it mixes patronage with capitalism.

      Linda: Yeah. So there’s a DAO called Yield Guild Games that I’ve been participating and active in, and there’ve been people in the Philippines who have been earning a living wage playing — like Jesse talked about you have these DAOs being able to own NFTs — and what they do, they’re a DAO of gamers: gamers from all over the world who are participating in these blockchain games.

      And there’s one really popular game called Axie Infinity — you have these like Pokemon-like creatures that battle each other, and each of these are NFTs — and you can battle and earn currency in this game. And sometimes these Pokemon creatures, like they might be too expensive because they’re so valuable. And so what we’ve actually seen in this DAO is players within this DAO leasing out NFTs to other players. It’s a really cool collective of people being able to join this group of gamers. And one thing that they’re doing right now is this DAO is investing in virtual land in the games that they’re playing, because they’re experts in these games themselves. And they’re actually developing like the land in these games as if…just in like a physical world of developing real estate and making it better (the idea is that they’re going to be just owning tons and tons of virtual land).

      Sonal: One quick thing — again, a DAO is a decentralized autonomous organization; people have often talked about cryptoeconomies over time enabling these sort of organizations because the history of the firm is very much entrenched in a physical world, not a digital world — but why do these things have to exist as a DAO; what’s the point of that? I’m asking because I’d want to know like, why a DAO specifically.

      Linda: So I don’t think everything has to be a DAO; there are plenty of times where a company makes a lot more sense.

      But, what’s really interesting about DAOs is there is a lot of more transparency — and so the funds that are managed by the DAO, it’s completely transparent where funds are being moved to and from, anyone can view the balance at any point in time. As you can imagine, a traditional company, you don’t have access to the balance sheet at all periods of times; and oftentimes, they’re just maybe released on a quarterly or annual basis.

      So the DAOs even the playing field, create more transparency; there’s lower barriers to entry in a lot of cases: You don’t even have to reveal your identity; it’d be really hard to join a company where no one knows who you actually are. That just fits very closely with the ethos of crypto of: global, open, kind of nature. <Jesse: Yup>

      Sonal: And by the way to be very clear, we don’t mean identity as in like anonymous, because you’re pseudonymous technically; like people can actually trace WHO you are without actually knowing who you are.

      The energy question

      So, now I’m going to have you guys break down even more misconceptions for me — like we talked about “just a JPEG, what’s so unique about a JPEG” — but there’s actually a lot more misconceptions, especially given recent buzz, all this commentary about “the energy, the energy, like minting is all this energy”.

      This is obviously an artifact of people thinking in terms of Bitcoin, which can be energy intensive; so, can you help clarify the energy question when it comes to NFTs?

      Linda: Well I think that there are a lot of misconceptions around that. So yeah, proof of work does require energy, but not every blockchain is proof of work. Proof of work involves physical miners actually verifying that these transactions happened. And so it’s just really energy intensive because you have to prove that you’re expending some sort of work, to produce this output.

      And so in Ethereum’s case, they’re migrating from proof of work to proof of stake — which is kind of equivalent of virtual mining – so, rather than spending let’s say $1000 on mining equipment, you’re taking that $1000 and locking it up into the system, and being randomly selected to verify based off the capital that you put in. So it’s just a virtual aspect of mining, and you don’t have to have the physical ones expanding energy.

      And then increasingly, we’re also having more movement towards Layer 2, like protocols built on top of Ethereum. Because people do want the be less energy intensive when it comes to verifying ownership on a blockchain. So there’s going to be less of that narrative, I think, going forward.

      Jesse: Even proof of work mining gets more of a bad rap than it deserves; it’s certainly true that it consumes a lot of energy. However, a lot of the miners who are doing the proof of work locate in areas where there is latent capacity — so renewable sources of energy that are untapped, for example like hydropower; there’s excess demand, well then it goes into mining. Like for example, there’s like natural gas emissions from oil fields; and that’s gas that is otherwise, it’s going into the atmosphere, but instead can be burned to produce proof of work proofs and earn Bitcoin.

      I’m not, you know, defending this practice. But I’m just noting that a lot of these emissions are either sort of latent, OR, a large part of the energy mix of proof of work mining is from renewables. And that again, is part of the conversation that’s under-discussed.

      Sonal: I am so glad you brought it up because the whole point of the show, again, is to give the nuance, that may or may not exist out there.

      Jesse: I also think a lot of the noise out there about the energy consumption of NFTs, really fails to take in the sort of relative measure of energy consumption more broadly.

      So, if you think about something like Art Basel, there’s a lot of very rich collectors who fly private to Art Basel every year in order to collect work. And I don’t know what the emissions of all those private jets is, but I would expect it’s a lot of CO2. And so to get into the game of quantifying the specific emissions of an artist’s work, I think is a very complex topic that’s sort of under-appreciated in 140 or 280 characters on Twitter when you say “Oh, this NFT caused X amount of CO2 emissions.” Well, what about all the freeports, what about all the private jets flying to Basel every year?

      So it’s a very nuanced topic, and I don’t think it’s fair to creators who are just using these new tools — which are becoming more and more efficient — to shut it down on the basis of this very headline-grabbing, relative value measure.

      Sonal: That’s fantastic. By the way I have been to Art Basel Miami, not the one in Switzerland, in 2006; I did not fly in a private jet, I was a grad student, I did not have that much money — but yes, I agree with you, that it would be very slippery slope.

      Linda: I also saw a tweet by Andrew Steinwold saying that actually, these digital art[s] are actually really environmentally friendly in that you’re not buying all these like physical supplies of like cotton for canvases, and wood for easels, and oil… And then you have all these shipping costs of moving this artwork to other people.

      There is always going to be aspects to anything that’s created, that you can always analyze and look at what is not environmentally friendly about it.

      Sonal: You’re absolutely right, and in fact, one of my absolute favorite artists, I bought a painting from her. I went to her show in New Orleans; I flew. She shipped the art to me afterward, it was such a process trying to bring it here, and the shipping — just even the materials to pack it, like all of it, it was intense and very complicated — and in fact, I had to hire someone to help me open the crate because it’s like screwed in, in wood. It was like not even possib- there was energy used to like take the thing out of this box. <Jesse chuckles> So, I agree with you. It’s a very tricky game to start comparing on that front.

      “Permissionless” innovation

      Okay! This idea of “permissionless”, you used that phrase a lot. If I were like a regulator and hearing that, I would freak out and be like, “Permissionless?! That means all kinds of bad behavior and actors and blah blah blah.” How would you address the concerns about things being permissionless — or even this idea that you know, you can’t even recover your key if it’s lost — there’s not like someone who’s holding that for you.

      Jesse: We can define it in the same way like cash is permissionless, right. Again, the wallet analogy is useful because if you lose your wallet, chances are you lose your cash and it’s gone. And similarly in the cash economy, you can buy all kinds of goods; they can be illegal goods, or they can be perfectly normal goods, and cash is used for both things. And so the same is true of cryptocurrencies, and I think the same is true of NFTs.

      There’s going to be bad actors right, there’s going to be people infringing on other people’s IP — and you know the legal system is going to have to step in and address those issues. However the benefits I think far outweigh any sort of negative or nefarious uses of the technology in that any developer can build new experiences around the way we consume media… again, when you contrast today where only the developers who work at Facebook or only the developers who work at Twitter can experiment, and innovate on the information that we see on those platforms… I think we’re in a much healthier state if EVERY developer in the world is free to innovate in a open way, without having to ask permission of these big platforms.

      Sonal: Right, that’s what you mean by permissionless. And by the way, it’s worth noting all those examples you cited — copyright infringement, IP, etc. — that’s pretty prevalent in the physical world, and you don’t often always have recourse (unless you’re Getty in doing this ridiculous royalty and provenance tracking).

      And, we’re talking about you actually have the solution baked into the very problem in the system here.

      Jesse: Yeah, in one sense, you could argue that blockchains actually make the job of forensics a lot easier because all the information is publicly accessible and available to anyone.

      Sonal: Our partner Katie Haun would obviously argue for that argument; I mean, at the DOJ that’s literally what she did!

      Jesse: Right, it’s all about finding the right balance where the bad actors can be addressed, and meanwhile the good actors are free to innovate.

      Corporate innovation & NFTs

      Sonal: Okay, so last thing. Can you guys give some just super quick practical considerations for startups or industry? I’d love to particularly hit mindsets, it doesn’t have to even be advice — for people who are consumers, people who are creators, and even institutional players.

      Linda: I find that just having conversations, and kind of plugging yourself into communities, and building in public is always really great to do in the crypto space. This space is still really early on, and people are all trying to figure everything out. So no one is a complete expert on what’s happening in NFTs and everyone’s very open to talking, collaborating. So never be afraid of asking questions; joining different communities on Discord, or Twitter, or wherever they’re chatting.

      Sonal: Big corporates and the big institutional banks and big DeFi players like banking and traditional players — they’re not the types to go into a Discord and try things out or have the mindsets you outlined. Any thoughts there, for that group?

      Linda: Traditional institutions can consider how NFTs could make things more efficient for themselves — so having these financial assets that everyone has to keep track of, might be really inefficient or costly. NFTs will enable this to just be a lot smoother of a process for them.

      So it may be worth looking into research — it doesn’t have to be digital art that they’re turning into NFTs; it could also just be unique financial assets that they have to manage themselves.

      Jesse: Yeah, big corporates and others participate in the markets for creative work, through various channels, a lot of companies work with creators and influencers on marketing and distribution. And NFTs offer a new channel for both of those things, right?

      I also think that, coming to be an owner of a creator/influencer’s work will be another way to gain their attention and potentially gain distribution through the lens of marketing. And that’s kind of an interesting idea.

      Sonal: One of the ones that I find very compelling is a new definition of employees in a modern era, where employees can be creators while working for a company and kind of get more ownership of their ideas — ‘cause traditionally it’s like very binary model; there’s not like a middle ground — and I wonder if that’s going to evolve through NFTs within companies, and even extending outside the borders of companies like in a classic open-innovation model.

      Jesse: So I think you’re touching on a really big idea — I would describe that as the ownership economy — where, in Silicon Valley, employees at startups get equity in a startup to align their incentives with the success of the company. And that model has worked really well for attracting the best talent to kind of work at startups. But it’s not been accessible to everyone, right, and as a result, the talent pool is not as big as it could be. And crypto kind of changes the game in that now it’s possible to send ownership value — whether an NFT or ownership of the bitcoin network — which you can now send that anywhere in the world instantly. And as a result, you can make ANYone an owner on the internet.

      And so I think this is a really profound idea where, it’s going to change the way that people come to work, in that they won’t have to go and become a full-time employee to earn some ownership value for the value that they contribute to the platform or service that they’re building or consuming.

      Sonal: Which reminds me of course of that famous Bill Joy quote that we all love which is that the smartest people in the world won’t ever all work for you. So, if you’re gonna embrace open innovation, open source, or extend your talent pool, that is a great way to give those employees quote-“ownership” — even if it’s fractional ownership, which is great.

      Jesse: Yeah, and NFTs make it accessible to everyone, not just technologists but consumers and creators as well.

      Sonal: Awesome you guys. So on this show — even though this a 3x explainer episode — I ask people to kind of give me a quick, short, you know “what’s your bottom-line” on this whole theme; give it to me.

      Jesse: I have a media background, so I love to fixate on a future where literally, every piece of media is incepted as an NFT — I’ve used this term a few times in the discussion, but I think what we’re building here is this universal library of media that’s programmable and where value flow is baked into the technology itself. And that’s just going to lead to a renaissance in online creativity, where the creators of the work are remunerated more fairly than they have in the sort of Web2 era.

      Linda: Yeah, there’s a lot of really exciting stuff happening in the NFT art space, and we have so many creative people coming in and it’s going to make crypto overall much better. But NFTs are also applicable to many industries where you track ownership, and currently have a middleman taking fees for that service. So, I expect there to be NFTs in all kinds of different industries like gaming and finance and healthcare and all kinds of other areas.

      Sonal: It’s an inevitable story of technology that you give people tools and things will happen — so it’ll be interesting to see what happens when we unlock that human ingenuity. You guys, thank you so much for joining this week’s episode, this 3x explainer episode of “16 Minutes”. Thank you so much, Linda and Jesse.

      Jesse: Thanks for having us.

      Linda: Thank you.

      • Jesse Walden is the founder and managing partner of Variant Fund, investing in crypto networks and founders. Previously, he was a partner at a16z Crypto and cofounder of Mediachain (acquired by Spotify).

      • Linda Xie is the co-founder of Scalar Capital, a crypto investment firm. She was previously a product manager at Coinbase and a portfolio risk analyst at AIG.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Semiconductor Shortage and the Global Supply Chain Squeeze

      Frank Chen and Zoran Basich

      In this week’s episode of 16 Minutes, our show where we talk about tech trends in the news, what’s hype/ what’s real, and where we are on the long arc of innovation, the topic is semiconductors – specifically, the ongoing global shortage that began last summer and has intensified in recent weeks. So much so, that the U.S. president signed an executive order just last week to address concerns around the shortage, calling for reviews of supply chains for critical sectors of the economy.

      Our expert is a16z Operating Partner Frank Chen, who led our research arm and has also joined past episodes about semiconductors on this show including one with Steven Sinofsky and Sonal in which they analyzed the ARM and Nvidia news.

      Frank joins a16z’s Zoran Basich to cover the bigger picture of the chip shortage including geopolitics, the pandemic, and complex worldwide supply-chain dynamics — all in almost exactly 16 minutes!

      Show Notes

      • How the semiconductor supply chain works [1:14], what caused the current shortage [3:10], and errors in forecasting that some companies made [4:49]
      • The impact of demand (types of chips needed by various industries) [5:54] and supply (cost of building factories, COVID disruptions, and the scarcity of raw materials) [8:24]
      • Geopolitical questions involved [11:55], including competition with China [13:58]
      • Thoughts about how the shortage will play out [15:14]

      Transcript

      Zoran: Welcome to this week’s episode of “16 Minutes,” our show where we talk about tech trends in the news, what’s hype, what’s real, and where we are in the long arc of innovation. The topic today is semiconductors, a topic that has been in the news a lot in different forms. Specifically, we discuss the ongoing global shortage that began last summer and has intensified in recent weeks. So much so that the U.S. president signed an executive order to address concerns around the shortage, calling for reviews of supply chains for critical sectors of the economy. 

      Joining us as our expert is a16z Operating Partner Frank Chen, who led our research arm, and has also joined past episodes about semiconductors on the show, including one with Steven Sinofsky and Sonal, in which they analyzed the Arm and Nvidia news.

      In this episode, we’ll cover the bigger picture of the chip shortage, including geopolitics, the pandemic, and several other factors, all in almost exactly 16 minutes. But first, Frank covers the immediate impact on the automotive industry, which was highlighted in the news of the executive order.

      The semiconductor supply chain

      Frank: A bunch of auto manufacturers, this includes GM, and Ford, and what used to be called Fiat Chrysler, now called Stellantis, all of these companies have basically shut down or slowed down their production of cars because they can’t get enough chips. There are very calibrated supply chains, tens of thousands of suppliers. And right now, what’s happening is, you know, Ford’s F-150, the best selling truck, is held hostage to a $0.25 cent semiconductor that all of a sudden they can’t get enough of.

      Zoran: So this news is really closely tied to the supply chain. That’s where the White House is focusing. Help us understand the landscape here, Frank. Who are all the players involved up and down the chain and how are they interrelated?

      Frank: The way that chips flow is that you have semiconductor vendors like Nvidia, and NXP, and Renesis, and Panasonic, Toshiba, Samsung, TI, and so on, they make the chips. Some of these companies make their own chips like Samsung and Toshiba, so they’re called integrated device manufacturers. Other companies like Qualcomm and Nvidia design the chips, but then use other companies like TSMC, or UMC, or Samsung, to actually make the chips. These design-only companies are called fabless chip companies. But in either case, you take the chips, you sell them to companies called tier one electronic systems providers. So these companies have names like Bosch, and Delphi, and Harman, and Denso, and Siemens, Continental. And then these companies in turn sell to what’s called the OEMs, the car manufacturers. That’s what we’d recognize as a car manufacturer. So a GM, a Ford, a Tesla, an Audi, a BMW, so on and so forth. Right? So that’s sort of the supply chain.

      You got a chip company selling to an electronic systems company. They might make the onboard entertainment system, they might make the anti-lock brake computer, or they might make the adaptive cruise control system. And then the OEMs assemble them into cars. And so, there is a lot of complexity.

      Causes of the current shortage

      Zoran: So how did the automakers get into this mess? What are some of the factors that caused this chip shortage?

      Frank: Let’s start with the demand side. So, three things. First, software is eating cars, which is to say that the percentage of a car that is electronics has been steadily increasing. So it used to be a car was rubber, plus glass, plus steel, and that was pretty much it. But now, in 2020, electronics are about 40% of the cost of a car. For a point of reference, in 2000, it was probably 18%. And, you know, these days, as you think about all of the cool safety features, right? Somebody’s in your blind zone, or adaptive cruise control, or automated stopping if there’s a pedestrian in front of you, right? All of those are obviously electronics.

      The second thing is, this was a classic case of under-forecasting. So what happened was, COVID hit, and everybody battened down the hatches. Everybody was like, oh, car demand is going to drop off the cliff here. And so we better be conservative and cut back all our orders. And what happened roughly at the same time was, there was a set of things that people wanted that demand went the other way, right? So think of Chromebooks, and laptops, and webcams, and everything that makes Zoom possible. All of this work at home stuff created a lot of demand for TVs, and computers, and so on. And so, the demand for semiconductors to feed those things sort of leapt into that vacuum that the car guys left. And then once we figured out that car sales weren’t going to be as dramatically impacted by COVID, they went back to the suppliers, and then they discovered the supply is not available. In other words, they got spoken for by the computers, and the cell phones, and the webcams, and IoT devices.

      Zoran: So there have been lots of headlines about this chip shortage, lots of angles to unpack, but let’s home in on the pandemic for a minute. How and why did this under-forecasting happen? And why specifically is that a problem when it comes to chips?

      Frank: I think it was human nature. Which is, COVID happened and you were like, gee, when will demand really ever come back to normal? And I don’t want to be the guy that over-ordered everything, right? Like, once the TPS reports come out, I’m a complete outlier because what kind of idiot would I be, if COVID happened and I was the guy who over-ordered everything by two orders of magnitude. The other thing is, it’s very long lead time to spin up a new semiconductor, in other words, to change the line so that it’s making your chip versus somebody else’s chip. This is measured in tens to dozens of weeks, right? So, 30 weeks. And so, you can’t turn it off and turn it on, on a dime, right? And so, when the orders got cancelled, the lines got retooled to build other chips. And so, to turn the line back on to build your chip, we could be talking the better part of a year, which is why most people think that the shortage is going to be another couple quarters before we’re done with it.

      Impact of increased demand

      Zoran: Okay, so on the demand side, you’ve mentioned the rise of electronics and cars. We talked about the forecasting mistakes. Let’s get a bit more into what’s hype and what’s real here. What else do we need to know to make sense of this?

      Frank: On the scale of things, car manufacturers aren’t the biggest customers for chips. So, by far, the biggest customer for chips is cell phone makers. Like, when Apple places an order, that’s an order, right? Like, that’s tens of millions, hundreds of millions, billions of components. And so, like, they’re more reliable as a customer. You can sort of see why if you are a chip manufacturer, like a Panasonic, or Toshiba, or Infineon, or STMicro, or TI — when Apple, or Samsung, or Dell, or HP comes to you and says, “I need more chips.” You say, “Of course. When do you need them?” Right? And then Ford, and Jaguar, and, you know, all of the other guys have to sort of kind of wait in line.

      And then the other thing that is true about car chips is, they can be harder to make, which is, their temperature range, their operating lifetime, the failure rates of these chips, right? So it’s like one thing for your Chromebook’s light sensor to go bad, it’s another thing altogether if, like, the radar that powers your adaptive cruise control goes bad. Like, that thing can’t go bad, right? Because you’re going to crash into somebody.

      Zoran: Yeah. And some of the chips that are used in cars are older generation chips. So, because they’re more expensive on the consumer electronics side, that’s another reason that they’re seen as more higher priority customers.

      Frank: So, yeah, two classes of chips, the older stuff, right, for the anti-lock brakes, and then the newer stuff for what the car industry roughly calls ADAS, right? So, the advanced driver assistance systems. One way to measure this is, what’s the nanometer process technology used to create it? And we’re headed towards seven, six, five nanometers. Think of that as sort of how fine is the etching on a semiconductor that defines the circuit path, right? And car manufacturers — some of their chips, you don’t need [a] 7, or 6, or 5 nanometer, you’re fine at 180. But the second class of chips that the car companies are increasingly buying are basically the exact same chips that go into a smartphone core, right? So if you think about machine learning, as we head towards autonomous, you know, they will be the most advanced chips, with a ton of transistors on them to do linear algebra, because that’s what machine learning demands. And so, this sort of emerging class of chips are exactly the same set of chips.

      Issues with global supply

      Zoran: Okay, so that’s the demand side. When the pandemic hit, automakers pulled back on their orders, while at the same time, demand was rising for consumer electronics. And the chip manufacturers turned to that segment of their customer base, and started producing for them. And it’s very hard to stop on a dime, and then take the automakers’ calls, who now suddenly are calling you and saying, “Hey, we want to restart our production.” That all takes a long time. So, let’s talk about supply. What happened on that side of the equation?

      Frank: So it turns out that the world has a fixed amount of manufacturing capability for semiconductors, which kind of seems bizarre, because we all know that software is eating the world. And, you know, the world has nearly infinite demand for chips. Most semiconductors are made outside of the United States, despite the fact that the United States dominates revenues for semiconductor design, through companies like Intel. But most of the fabs, which is the factories that make semiconductors, are overseas. But it turns out these fabs are incredibly expensive and relatively low margin to build. Think of a factory that might cost $10 billion to build, and it’s obsolete in five years. The rate at which semiconductor fabrication changes is so fast that, like, all of the equipment that you just bought from Applied Materials and JLA-Tencor, like, that thing is going to be obsolete in five years.

      And so you have these incredibly expensive factories that depreciate very, very fast. The equipment in it sort of needs to get replenished very, very quickly. And so, the industry’s kind of rewarded companies that are called fabless designers. In other words, they’re companies that design chips, but don’t actually make them. So the fabless designers that we all know are companies like Qualcomm, and Broadcom, AMD, Nvidia, Apple itself, right — all of the, like, super awesome chips that they design, they don’t actually make. They go to semiconductor fabs, the largest of which is the Taiwan Semiconductor Company, that actually operates the factories. The top three countries that actually make chips, South Korea, Taiwan, and Japan, they have all of the factories because fabless is rewarded by the investing community. You’d much rather have the higher profit margins of a Qualcomm than the lower profit margins of a TSMC, the Taiwan Semiconductor Company.

      Zoran: Okay, so you have this issue of how incredibly expensive it is and how frequently you have to accommodate the new technology they have to build. What are some of the other supply factors?

      Frank: Well, like every other manufacturing facility, the semiconductor industry got hit by COVID itself. People need to be in the factories feeding the wafers, and doing quality control. And so there was some slowdown as a result of that. Now, the good news is, South Korea and Taiwan are the biggest manufacturers, and they were much less exposed to COVID because they were very aggressive with their lockdowns. But there are other things about semiconductor manufacturing. So, Taiwan is going through a drought, just like California went through a drought. And it turns out, you need a lot of water to make chips. TSMC’s daily water consumption is 156,000 tons a day. In the northern part of Taiwan, where these factories sit, that amount of water is 10% of the region’s daily supply of water.

      And so, you have all of these weird supply things. You have, like, raw material shortages. We’ve got the COVID hit. We’ve got the very, very small number of fabs, just because they’re expensive, and sort of the financial community willingness to fund these super expensive factories is sort of low.

      Geopolitics of semiconductors

      Zoran: So we’ve talked about the factors like drought, the economics of chip production, the pandemic, obviously. Now, where do the geopolitics come in? How big a deal is that, really?

      Frank: Yeah. So, geopolitics is a big deal. So, you’ll remember that the Trump administration basically forbid American companies from buying from Huawei. And then they later extended it to, you know, requiring a license to sell to Huawei. And then there are licenses that the semiconductor tool chain now has to apply for in order to sell to Chinese companies. And so, if there were no politics, you would do what manufacturing has done for the last four decades, which is, you’d fire up China. You’d make it possible for Chinese fabs, not Taiwanese ones, not South Korean ones, not Japanese ones, to bloom. But the problem there is that the United States, rightfully so, wants to be a little careful about what kind of semiconductor manufacturing equipment they will sell. Because the worry is that, they’ll buy it, they’ll reverse engineer it, they’ll infringe on the intellectual property. And lo and behold, they can make their own semiconductors. And by the way, that is China’s explicit goal, which is that they want to have the number one semiconductor design and manufacturing industry in the world by 2030.

      And so, now the whole thing is a geopolitical dance. Like, you could imagine any administration, this is not a red or a blue issue, saying, like, we’re not sure that we want to sell China the equipment to make the most advanced semiconductors.

      Zoran: It’s interesting because, you know, there’s this perfect storm that happened with the pandemic, and also some other factors, which we haven’t mentioned. You know, there were a couple of fires in Japanese factories that had a negative impact on supply. Even in Texas, there were a couple of factories that because of the recent cold spell, had to shut down for some time. So all these things kind of just built upon each other to create this somewhat perfect storm. And so that created this instant problem, but it also highlighted this larger problem [that] needs to be addressed. So what’s the bigger picture about what this means for innovation, given these geopolitical pressures?

      Frank: Yeah. I mean, one of the big things that the industry is asking itself is, you know, China’s ambition to be number one, can they get there? And they don’t want to stop at just the chips, right? They want to be the complete, fully vertically integrated stack. So if you think of the vertically integrated stack, it’s sort of iOS running on top of a bunch of chips, with iOS. Or if you think about Windows, running on Intel. Like, in the next 10 years, will there be a Chinese operating system running on a Chinese chip, right, with all of the motherboard, etc., etc., design done by China. And if that were the case, will we really have two competing world ecosystems? It kind of reminds me of, like, the early days of communication. There was Docomo in Japan, and then there was Minitel in France. And like, they didn’t talk to each other. They were just little islands. And we knew that that world wasn’t good. What the world wanted was the internet, the connection of networks.

      But if we go back to this sort of geopolitically motivated desire to have your country own the factors of production in a completely integrated vertical stack, hardware and software, then we might go back to the bad old days, where compatibility was hard. And, you know, we sort of were kind of wasting “R&D” dollars building the exact same thing, just in slightly different ways.

      When the shortage may end

      Zoran: So we have the White House, President Biden, calling for this review. Short term, what’s going to happen here? And how long will this last?

      Frank: So most people are forecasting a couple more months, maybe quarters of pain. I don’t think, unless something surprising happens, I don’t think this is going to last that long.

      Zoran: I thought the solution was more manufacturing facilities need to be built in order for this truly to become solved. So how is this going to solve in the short term?

      Frank: You know, I don’t think it’s going to take, like, a brand new fab to unlock the current snarl that we’re in. So in the short term, you know, look, we’re not going to be at post-COVID highs on webcam orders forever, right? Like, they will go back to normal. And so the heat on alternative demand will sort of cool some. And then, you know, we’ve got the automaker chips in the queue now, right? And so, like, eventually, things will sort out. We’ve always had component shocks in the tech ecosystem. It’s just every now and then, we’ll hit a bad one. This one’s a pretty bad one because we have so many car factories making very, very expensive products, stymied by their $1 semiconductor being missing.

      Zoran: And that brings us right back to the news, full circle. So, let’s go to our bottom line, Frank. What are your takeaways and final thoughts on the topic?

      Frank: We have some soul searching to do about what the shape of our supply chains ought to look like and how much they should reflect the geopolitics of the time, or if technology wants to and should be a country-independent thing. Where, you know, the best ideas, meritocracy, take the day, as opposed to, we’re going to have the U.S.-led tech stack, and then a Chinese-led tech stack, and then they don’t really speak with each other, and view each other with mutual suspicion. The supply chain for technology has always had shocks. Because software is eating everything, these shocks are now rippling beyond technology.

      Zoran: Frank, thanks so much for being with us today.

      Frank: All right. Thanks, Zoran.

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

      Assembling an Egg

      Vineeta Agarwala, Justin Larkin, Judy Savitskaya, and Lauren Richardson

      On this episode of the Bio Eats World Journal Club, we explore the very compelling question of whether we can use our understanding of developmental biology to create oocytes (aka eggs or female gametes) from stem cells in the lab. If possible, this could be on par with the development of in vitro fertilization in terms of extending fertility. But creating an oocyte from a stem cell has some unique and high-stakes challenges. Host Lauren Richardson is joined by a16z general partner Vineeta Agarwala and deal partners Judy Savitskaya and Justin Larkin to discuss the research article “Reconstitution of the oocyte transcriptional network with transcription factors” by Nobuhiko Hamazaki, Hirohisa Kyogoku, Hiromitsu Araki, Fumihito Miura, Chisako Horikawa, Norio Hamada, So Shimamoto, Orie Hikabe, Kinichi Nakashima, Tomoya S. Kitajima, Takashi Ito, Harry G. Leitch and Katsuhiko Hayashi, published in Nature, which makes a big step towards this goal. The conversation covers which aspects of oocyte biology the authors were able to replicate, which they were not, and where we think this field might be heading.

      Show Notes

      • A brief discussion of the biology [3:24] and terminology [5:27]
      • Potential uses for oocyte biology [7:34], what it would take to convert cell types [10:36], and the development of the oocyte [12:36]
      • Limitations of this study [16:59] and the future of oocyte biology [19:25]

      Transcript

      Hanne: Hi, I’m Hanne.

      Lauren: And I’m Lauren, and this is the Bio Eats World Journal Club, where we discuss breakthrough scientific research, the new opportunities it presents, and how to take it from paper to practice.

      Hanne: So, Lauren, you’ve titled this episode “Assembling an Egg,” but I’m going to guess we aren’t discussing your favorite breakfast recipes today.

      Lauren: Hanne, you know me well, nope. Today, we are exploring the very compelling question of whether we can use our understanding of developmental biology to create oocytes, AKA eggs — you know, the female counterpart of sperm — from stem cells in the lab.

      Hanne: Okay. So, refresh my memory. Do we have stem cells in our adult human bodies that could be used to turn into eggs?

      Lauren: Yes and no. Adults have stem cells, but their ability to turn into other types of cells is limited. For example, we have stem cells in our bone marrow, but they can only produce the different types of blood cells, but scientists now know how to turn some of our cells back into stem cells, which are called induced pluripotent stem cells, or iPS cells. If possible, this adult cell to iPS cell to egg cell transformation could be on par with the development of in vitro fertilization in terms of extending fertility, but creating an oocyte by this path is tricky. So, I’ve gathered some of our colleagues, a16z general partner Vineeta Agarwala, and deal partners Judy Savitskaya and Justin Larkin, to discuss a recent research article published in Nature, by Hamazaki et al., that makes a big step towards this goal. We discuss what aspects of oocyte biology the authors were able to replicate, which they were not, and where we think this field might be heading. We start with Vineeta describing the state of innovation in fertility. 

      Vineeta: Fertility, even just within the U.S., is approaching a $10 billion industry, and the majority of innovation that we see happening in the space is related to care delivery. How can we expand access to fertility treatments to couples who need them? That’s an important problem, but at the same time, it makes us wonder whether there are biological breakthroughs that could change the face of the industry even more than care delivery technology might.

      Justin: Yeah. For so many aspects of reproductive therapies and infertility challenges, the egg or the oocyte is often the critical limiting step, both from an ability to actually get to a healthy pregnancy, but also in the process of the patient journey. Having seen numerous people go through the IVF process and other kinds of iterations of IVF, it’s often just so hard from a patient journey perspective — literally being painful, takes lots of time, it’s expensive. And so, the thought of being able to fundamentally change access to oocytes, and eggs that potentially could be grown in vitro in ways that cut the process down and make it more affordable, has the potential, I think, to not only open up reproductive technology and therapies to a broader population, but to make that care experience just fundamentally different for patients. And so, seeing the early innings of that potentially be suggested in this paper was fascinating to me.

      Judy: Vineeta and I were actually pretty surprised the other day when we realized that the most expensive part of IVF, the single biggest line item, is the drugs that are required for egg retrieval. So, this is a way to just get around that entire process.

      Overview of oocyte biology

      Lauren:  Right. Could you unpack that? How is this solving the problem?

      Vineeta: Women are born with a couple of million potential eggs, and as we age, we lose those eggs, and we don’t make more. And this paper provides this suggestion of a way to engineer, from pluripotent stem cells, oocytes that can potentially be fertilized and give rise to a new being, without being dependent on that very constrained, constantly dwindling supply of oocytes that we’re born with as women.

      Lauren: Right. I think that is such an interesting idea, that when your baby is still in you, it has its full complement of eggs that it’s ever going to have. Females don’t make additional eggs after birth, so you have to work with what you have. As Judy said, it’s very expensive to retrieve eggs from a woman. So, this paper asks a really tantalizing question of — can we make new eggs from cells after birth, from your own cells after you’ve been born?

      Vineeta: I would think of it as a replacement for egg freezing, is one way to contextualize where this could, hypothetically, fit into the industry. Today, women are freezing eggs in their 20s so that they may access them in their 30s and 40s or beyond, and this would provide an alternative to having to undertake that process of retrieval, freezing, and hoping that the eggs are usable.

      Justin: Often in those retrievals, you’re only able to get — you know, let’s call it 1, 2, 3, sometimes if you’re lucky, more than that — but a limited number of oocytes, and so you have a limited number of shots on goal on potentially having a successful pregnancy. Where I think what got me excited about this paper is this idea of potentially having more opportunities [for] a successful pregnancy that aren’t as constrained by the cost and kind of care experience today that drives the typical IVF process that we alluded to earlier.

      Lauren: We’ve thrown a couple different terms around. We’ve thrown around oocyte, we’ve talked about eggs. Let’s define this. What is an oocyte, exactly?

      Justin: Yeah. An oocyte is not a static thing, and it goes through a number of different developmental phases, starting from an undifferentiated cell, ultimately to being a true oocyte, which is what we would call a gamete or female sex cell. 

      The journey to get there is quite a fascinating one. As you alluded to, it starts, actually, in utero. Then as it gets further along in its development, it starts out as what we call a primary oocyte. And as a primary oocyte, it still has the deployed genetic material. Then it starts meiosis, which is the process of going from being deployed and having two copies of chromosomes, to being haploid or just having a single copy. And in that process, it actually arrests and will stay in that arrested state until that female hits puberty, and then that full mitotic process isn’t completed until after ovulation and even with fertilization. 

      So, it’s highly dynamic, and I think the critical piece here is that there’s a lot of gene regulation that’s going on throughout, which opens up the opportunity for research studies like this to really define what are the regulators of that process. Can we recapitulate that in vitro setting to potentially have some of the applications that we talked about earlier?

      Lauren: Yeah. I definitely was doing my background reading in preparation for this conversation, and really appreciating all those different steps that happen during — in utero. What happens before puberty, what happens during puberty, what happens during ovulation, what happens at fertilization — there’s all of these different steps, all these different developmental changes that all of them seem to carry the name oocyte.

      Justin: One other nuance that I think is important, too, is just that the oocyte in isolation isn’t necessarily enough, right? There has to be what they call a follicle, which is a set of cells that surrounds it. It’s very hormonally active, a lot of cell signaling and transcription factors are produced in that process. And we start to see early recapitulations of that in this study. But in vivo, inside of the female, there’s a lot of other, kind of, cellular activity that goes on around the oocyte cell itself that has a lot of hormonal impact downstream.

      Potential uses in healthcare

      Lauren: Yeah. That’s a really good point, that it’s very dependent and has a very complex interaction with the ovary and the cells of the ovary. So, on previous episodes of “Journal Club,” we have talked about converting one cell type to another. For example, we’ve talked about converting stem cells into the cells of the pancreas that produce insulin. But what are some of the particular hurdles in creating an oocyte that you wouldn’t have to deal with in just converting a stem cell into, say, a pancreatic cell?

      Vineeta: One way to think about the second-order challenge here is that you have to not only get to a primary oocyte, which has grown and has the right differentiation to at least have started on the path of egg generation, so to speak. But beyond being a primary oocyte, you have to get to a secondary oocyte. And the difference is that you have to undergo the whole process of meiosis, which is how we generate genetic diversity. 

      And so, that secondary oocyte is a pretty complicated thing to make. Not only because its genome has to have halved in a very unique way, by a very unique process, but a lot of epigenetic signaling that then goes on to determine gene expression networks in the subsequent fertilized embryo are thought to be driven by a program set in that oocyte. So, some genes are expressed off of DNA you got from your mom, some genes are expressed off of DNA you got from your dad, and not vice versa. A lot of that epigenetic programming is thought to stem from a pattern that’s encoded in the secondary oocyte.

      Judy: Another major difference between this and other cell types that you could potentially reprogram is just the morphology. So, oocytes are the largest cell that we have in humanity. The size and shape is just really different from what you would expect for other cell types.

      Lauren: Right. So, in my previous example of turning a stem cell into a pancreatic cell, there might be morphological differences between those two cell types, but they aren’t necessarily at the scale that you would see in a stem cell to an oocyte. And, fundamentally, you aren’t changing the genome when you’re doing this transition from a stem cell to a pancreatic cell — the genome stays exactly the same. You’re just changing what genes are expressed. 

      When you’re creating an oocyte, you’re halving the number of chromosomes that you have in a cell. So, a mature oocyte only has half the number of chromosomes that a normal cell does, and that’s because it meets with a sperm that has half the number of chromosomes that a normal cell does. And now, when they fuse and fertilize, now you get the full complement. So, having to do both of those steps is far more challenging, and represents an additional hurdle that you have when creating these iPS or stem cell-derived oocytes. 

      Lauren: So, let’s talk about how you convert one cell type into another. What are, kind of, the — how do we think about how cell fate is controlled?

      Judy: So, the way that we think about cell types is often which transcription factors are present. And those transcription factors are proteins that will bind to the DNA and basically cause the production of RNA from a given genomic locus, so from a given gene in the DNA. So, transcription factors are used as the sort of master regulators of a cell type, where one transcription factor might be relevant. 

      Just to make it super simple, let’s say a certain transcription factor is relevant for neurons, and another one is relevant for hepatocytes. You would expect that all of the genes related to hepatocyte function and development are going to be controlled by that hepatocyte transcription factor. And in the neuron cell, that transcription factor is not present, so none of those relevant genes are made. That’s an oversimplification, but that’s kind of the idea of how these genetic regulatory networks work.

      Lauren: Right. And so, when we’re thinking about in the lab, in the clinic, if we want to convert one cell type into another, we can express these transcription factors, which then leads to the expression of all their downstream genes. And then that, kind of, reprograms the cell and says, “We’re a hepatocyte now, we’re a neuron now, we have this transcription factor that’s driving this — let’s call [it] the gene regulatory network.” And that guides the cell to a specific identity. 

      So, in this paper, what they’re doing is they’re trying to identify the transcription factors that govern this development of the oocyte so that they can take those transcription factors, express them in a stem cell, and then encourage that stem cell to become an oocyte. So, with that in mind, let’s start with how the authors identified these key genes — these transcription factors that are involved in oocyte development.

      The development of the oocyte

      Vineeta: My understanding is that they did whole transcriptome profiling of cells at different stages in mouse oocyte development. So, basically, compared the differential expression of lots and lots of genes at each of those different time points, and constructed a network analysis to nominate specifically not just genes, but regulatory genes. And they used a bioinformatic GO search to get to the subset of transcription factors that they believed were driving the evolution of the transcriptome through the differentiation process.

      Judy: Yeah. I think a lot of those genes were already known from studies that had done in vivo work, so I think it’s important to make that distinction. You can do this work in vivo, which means taking oocytes at different stages of development in a mouse and actually measuring the transcriptome there. Or, they created a sort of — an organoid model, for lack of a better word — where they put cells into an environment that is similar to what they would experience during oogenesis, and measured the transcriptome at different points in that process. 

      And so, I think the purpose of this was partially to find the genes that are relevant, but also partially to identify the period of time within this model system that should map onto the period of time in in vivo oogenesis. To be able to, basically, run their experiments, and know the right time period to look at the cells.

      Lauren: I think that’s a very good point. It was both a — can we identify the correct transcription factors? But can we also validate this very handy in vitro system that we can then use to do further study?

      Vineeta: And another way to think about it is — in biology, we talk a lot about necessity and sufficiency, and I actually think they did a better job with necessity than sufficiency. They proved that if you knock out any of these top-nominated transcription factors, that you can’t get differentiation past a certain stage. And you really need this set of factors to be expressed in order to get to a primary oocyte. Sufficiency is a much higher bar, right? You have to prove that the thing you got, the primary oocyte you got, can then go on to do all of the things that you expect it to do. And I would say it’s almost impossible to prove until you’ve generated the end state of, like, a mouse baby. So, they make some progress towards sufficiency, but less.

      Judy: There’s also a sort of curious result that I would like to, I don’t know — talk to the authors about and maybe understand a little bit more. Which is that they find these eight genes that they collectively called PPT 8, and they do all of their experiments with these eight genes. But there’s a paragraph in the paper that talks about how they found a subset of four that is sufficient to get the same phenotype you see. But if you have some subset of five, six, or seven that contains that subset of four, it doesn’t necessarily mean that that’s going to work. 

      So, there’s sort of this — they are sufficient, but then some other part of those eight — that set of eight — has some interaction with that set of four, such that they didn’t trust that the set of four was truly sufficient. So, there’s some complexity going on there that they’ve definitely moved on from by just using the set of eight.

      Lauren: Yeah. There does seem to be maybe a higher level of regulation that hasn’t been elucidated yet.

      Justin: A couple other things I thought were interesting, too, is that they had to also expose them to just some somatic cells from the ovary to get the development to take place. And so, I think it, again, reiterates the point that while these transcription factors are likely necessary, there’s also some dynamic signaling that’s happening from somatic cells that surround, as opposed to just being purely driven by those transcription factors. 

      One other thing that also caught my attention is the timeline that this all took place in. If you look at what’s happening in vivo, this is usually happening over seven or eight days inside the mouse model. But within the in vitro model, they saw this happen over a couple of days. And so, this is likely a necessary set of transcription factors and regulators, but there are likely other regulators — potentially checkpoints or others — that are absent in the system that’s allowing it to run through this process on a much more accelerated timeline. And potentially could explain some of the other issues that they see downstream with having the morphological appearance of an oocyte, but a lot of the functional aspects of it didn’t quite get that.

      Limitations of the current research

      Lauren: So, what elements of a functional oocyte were they able to recapitulate with these oocyte-like cells, and what’s missing?

      Judy: So, what they did successfully show was growth, the morphology. They also showed a couple of expressions of certain factors that we associate with oocytes, but what they didn’t show is that there’s the right dynamics surrounding the DNA. For example, meiosis didn’t occur, which — it’s also not something that they were aiming for, so I don’t think that’s the bar that we should hold them to, but there was no meiosis. So, what you end up with in the end is a cell with a lot of extra DNA in it. And then the other really important piece is that the methylation pattern is incorrect. It’s completely different from what you would expect in an oocyte.

      Lauren: The methylation — that’s one of the key epigenetic modifications. So, that governs how the chromosomes are packaged, and that leads to how accessible certain genes are to be turned off and on at particular rates.

      Vineeta: The maternal and paternal imprinting, the mechanism by which is most commonly methylation of the DNA, is really important for health and disease. We know now of many different disease states that are actually attributable to incorrect maternal or paternal imprinting. And so, it’s not a minor issue that methylation wasn’t solved, and one that we’d have to pay a lot more attention to as this research advances.

      Lauren: Yes, that’s a really good point. So, the real test — the final test of whether you got an oocyte or not — would be to fertilize it with a sperm and to grow a new being — in this case, a mouse pup — up from that. In this paper, they tried that, but it didn’t quite work. What was the result of this experiment?

      Justin: So, what they saw — and somewhat not surprisingly — is that when they did fertilize it, a very small percentage of the cells actually went on to cleave at all, and even of those that did, very few made it beyond the two or four-cell division. Which, a lot of the early cleavage in cells is dependent on having that haploid one single set of chromosomes. And so, in this scenario where they weren’t able to achieve meiosis, which means that they weren’t able to go in with a haploid cell, it’s unsurprising that when the cell was fertilized, it resulted ultimately in non-viable embryos — given that the chromosomal count mix is not consistent with traditional fertilization.

      The future of oocyte biology

      Lauren: Yeah. So, the paper makes some really great advances in our understanding of how an oocyte develops, what the gene networks are, what the transcription factors are that are regulating these — but there are still a lot of mysteries, and still a lot left to study in this process. When you think about these unanswered questions at the end of the paper, what are the questions that interest you? Where are you interested in seeing this work go next?

      Justin: I think for me, when I look at this paper, there are two, kind of, key avenues you could take. One is to look at this as, “Can we use the oocyte as a structural scaffold for other scientific applications?” And we see this happening already today, where enucleated oocytes, or rather the oocyte without the nucleus of the DNA material, are used in applications for mitochondrial disease to other potential therapeutic applications. Those are relatively limited. 

      And so, for me, the biggest question that this tees up is, “What are the next steps that need to be taken to really understand how we get the nuclear — how we get the meiosis portion of this correct? Because if we’re eventually going to reach, kind of, the vision that we outlined at the beginning — of having this be a critical asset and enabling greater access to fertility treatments — it’s really an absolute necessity and table stake in order for this to progress.

      Judy: Yeah, I totally agree with Justin. At the end, the authors say that this is a potential — a potential use case here is somatic cell nuclear transfer, which is another way of saying cloning. I think that there’s not that much need for this kind of a solution for those applications. I think we should really see this as a stepping stone toward [an] entirely ex vivo generation of an oocyte for the purpose of in vitro fertilization.

      Vineeta: Yeah. I think one of the things that’s hardest about this particular — the fertility use case of stem cell research is that, presumably, the parents of a prospective child want to see their genome represented in the progeny. A lot of other applications of stem cell research don’t actually have that requirement, especially if you can design creative ways to avoid immunosuppression and to create immune cloaking of a stem cell-derived therapy, and so on. You might actually envision in a lot of other fields an off-the-shelf stem cell-derived or iPSC-derived cell therapy that can be really therapeutic for a lot of patients with different diseases. 

      Here, we can’t have that, or at least it doesn’t solve some part of the core fertility challenge. And so, because you’re so dependent on actually running this process on a case-by-case basis with each couple, I think we just have an even further way to go on this. It’s not like you could create a bank at some point, and then differentiate them from that every time you need to spin up a new embryo. You really have to get the whole process right end-to-end from the point of a patient-specific iPSC cell line. And that’s really hard.

      Lauren: There’s not to be an allogenic option.

      Vineeta: Exactly. There’s no allo-embryo to be had here.

      Justin: And the bottom line for me is — in thinking of the ultimate translation of this, we’re obviously in the earliest of early innings in terms of this actually translating to being, kind of, that holy grail for fertility and the fertility treatments that we talked about earlier. I think this does clarify a lot of our understanding about what it takes to create a structurally similar cell to an oocyte. But ultimately, to reach the broader vision that we want, there’s still a lot of work that needs to be done.

      Judy: This is actually a really important step for developmental biology as well, because I think it’s one thing to do descriptive research, where you understand an existing system and you characterize all the pieces of it, and it’s an entirely different level of understanding when you can actually rebuild it. So, I think there’s the phrase Feynman says: “You don’t really understand something until you can build it,” or some variation on that. And so, this is a perfect example of a paper that’s using building to get to an understanding that is deeper than what we had before.

      Lauren: I think that’s a perfect note to end on. Justin, Judy, Vineeta, thank you for joining me on “Journal Club” today.

      Vineeta: Thank you, Lauren.

      Judy: Thanks, Lauren.

      Justin: Thanks, Lauren. Thanks, Judy. Thanks, Vineeta. This was fun.

      Lauren: And that’s it for “Journal Club” this week. If you enjoyed this episode, please subscribe, rate, and review wherever you listen to the podcast. And to learn more about how biology is technology, subscribe to our newsletter at a16z.com/newsletters.

      • Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

      • Justin Larkin is a deal partner at a16z where he focuses on healthcare technology companies. Prior to joining the firm, he led strategy and operations at Verily and cofounded Wellsheet.

      • Judy Savitskaya is a deal partner at a16z where she focuses on bio companies. Previously, she worked on synthetic biology research at UC Berkeley and was a computational modeling and neuronal networks researcher.

      • Lauren Richardson

      Value Versus Volume (in Healthcare)

      Todd Park, Vijay Pande, and Hanne Winarsky

      The way we pay for healthcare in the US has long been by fee-for-service: per doctor visit, per test, per surgery, per hospital stay. But that system has led to rapidly escalating volumes of services and cost to the system—without actually improving outcomes. What if we shifted everything towards paying for value—and outcomes—instead? In this episode, Todd Park, co-founder and executive chairman of Devoted Health, and formerly Chief Technology Officer and technology advisor for President Barack Obama; a16z General Partner Vijay Pande; and Bio Eats World host Hanne Winarsky—talk all about the megatrend of value-based care, and how it is redefining healthcare itself. Why is now the moment for this massive shift? How do we implement it? What does it mean for doctors and patients, insurers and policymakers? What is tech’s role in making it possible, and what’s the business model and incentive for creating value?

      Show Notes

      • Discussion of what value-based care is [1:20], and how it emerged as an alternative to the current system [4:35]
      • The importance of targeting healthcare [7:11], and how to scale the value-based system [13:11]
      • Why there is no “silver bullet” to fix the current system [16:00], but how we might shift toward a new paradigm [24:13]
      • Discussion of first steps that may be taken [31:06] and what it would look like to rebuild the healthcare system from scratch [34:00]

      Transcript

      Lauren: Hi, I’m Lauren.

      Hanne: And I’m Hanne. And this is “Bio Eats World,” our show where we talk about all the ways that our ability to engineer biology and reengineer healthcare is transforming the future. And when it comes to reengineering healthcare, there’s one concept that gets a whole lot of airtime — the concept of value-based care.

      Lauren: Value-based care is a term that we’ve thrown around on many different episodes about how the healthcare system is evolving, but we’ve never really gone straight to the heart of the matter.

      Hanne: So, that’s what we do in this episode with Todd Park, co-founder and executive chairman of Devoted Health, and formerly chief technology officer and tech advisor for President Barack Obama — along with a16z general partner, Vijay Pandey, and me, Hanne. So, what exactly is this big mega trend of value-based care all about? And how is it redefining what we think of as medicine, treatments and healthcare? What does it mean for doctors and patients, insurers, and policymakers? Why is now the moment for this big shift, and what exactly is tech’s role in it?

      Defining value-based care

      Hanne: We hear the term value-based care thrown around an awful lot, but we’ve never really talked about what that means. So, this conversation is really about what is value-based care? How do we implement it? Why is it better? And what is technology’s role in that? So, maybe we could just start with — how is it different from the healthcare system today…

      Todd: Oh, it’s really different from how the healthcare system generally works today, famously, or infamously.

      Vijay: Even aspirationally. I mean, when you think about how the plumbing works.

      Todd: That’s right. The root cause, honestly, is how the healthcare system is paid for. So, historically, U.S. healthcare has been paid for in what’s called, “a fee-for-service.” Frank, right. So, doctors, hospitals, and healthcare providers are paid per thing they do — like, per doctor visit, per test, per surgery, per hospital stay. That has led to a situation where we have rapidly escalating volumes of services being delivered. But, unfortunately, we don’t actually have commensurately improving outcomes. We spend the most per capita of any country in the world. We rank at the bottom of the developed world on metrics like avoidable death, preventable death, adverse events, healthy life expectancy — and that’s because we fundamentally have a pay-for-volume payment system. 

      If you want to change the situation, then change how healthcare is paid for. Basically, [we should] move away from pay for pure volume to engage in value-based payment — which, in a nutshell, is a payment system that actually financially supports, aids, and abets right care, right place, right time. Value-based care, to me, is the right care, including non-clinical support, delivered in a consistent, coordinated, and proactive way that both improves outcomes and lowers costs, [thus] saving money.

      Vijay: It’s interesting to think about how this came about historically. Basically, [it was] a perk given to workers to help them stay with a given employer instead of somebody else. And so, that perk was, like, “We’ll get together, and we’ll pay for your medical bills or pay for things.” It’s kind of akin to, like, I don’t know, like if we gave a perk to say, “We’re going to pay for your plumber bills,” or, “If you have some major catastrophic problem, we’ll pay for the plumbing.” But that doesn’t mean we’re going to keep track of your house, or try to see if your plumbing is in good shape to avoid the problems. We’re not paying for that. That was never part of the plan.

      Hanne: We’re not paying for copper pipes. We’re paying for when something really goes wrong.

      Vijay: We’re paying for when the pipes burst. But we’re not paying for maintaining your house. That’s something that was always assumed to be on the patient side, so to speak. The thing is, we’ve gotten very good at trying to come up with therapies for cancer, for stroke, for massive heart disease. But, what we’ve come to realize is that that’s actually more expensive, because if we wait that long, those therapies can work. But [these therapies are] painful in many different ways, emotionally and physically for the patient. We want to get there before the pipes burst. Think about the house and the overall health of it.

      Hanne: So, can we talk about when this concept started to emerge? Why was there this gradual dawning of realization that [value-based care] was a better north star to orient towards, or would everyone say this is a better north star?

      Todd: I think in the last five years, the move to value-based payment and care has gone from, “Is it going to happen?” to, “It’s going to happen. It’s just a question of how fast.” It’s an idea that’s been around for decades. I mean, it goes in the category of a super obvious idea.

      Hanne: Right. Right.

      Todd: For example, if you have diabetes, or hypertension, or congestive heart failure, there’s an incredibly well-known best practice pathway in the form of medication regimes that get adjusted, based on each patient’s evolving situation. Along with very basic coaching on diet and activity. It’s very, very straightforward to execute. If I’m a primary care doctor being paid fee-for-service, and I’ve got a patient in front of me who has diabetes and hypertension, and I want to spend an hour with that person — to really help educate them about their condition, and really get into it, in terms of coaching and what to do, in terms of coordinating their care and providing them with the right support — I literally can’t afford to do it. Because I have to, in that same hour, see another three patients and get paid the fee per those services to stay alive. It is a bankrupting action for me to actually spend the extra hour; it is far more difficult to actually financially support and execute those pathways than it should be.

      Poorly controlled chronic illness is the greatest single driver of more serious events like heart attacks, strokes, kidney disease, eye and nerve damage, and vascular disease. And it’s just nuts. We as a country don’t do these incredibly basic things to keep people healthy during these chronic conditions. In addition to these patients being cared for the way they should be, you will save so much money, because I’m going to actually save you one, two, three hospitalizations that cost $20,000 each. We’ve invested so much in developing incredibly advanced therapies for acute situations, but we aren’t doing the basics. Because systemic execution of the basics, with process control and improvement that every other industry would find routine — it really does require value-based payment and value-based care as the paradigm to make that happen.

      Vijay: Yeah, otherwise, there’s no incentive.

      Todd. Right.

      Getting patients the care they need

      Hanne: Todd, when you say the right care at the right place and the right time, can you talk about what exactly that means? And what would that mean on the entire country model?

      Todd: Maybe the simplest way to explain it is, as opposed to me as a primary care doctor being paid X dollars to see people for 15 minutes, I am given what’s, in effect, a global budget for all medical spending — physician, drugs, tests, surgeries, and hospital stays — for my patients. It’s risk adjusted. If I’ve got a patient panel that has significantly sicker patients, then my budget’s higher, because essentially, the budget’s set to be equal to what the healthcare system has generally been spending to care for folks in a situation where patients have not been getting — in a systematic, universal way — right care, right place, right time. Highly prevention-oriented care. So, I have this budget that I’m working off of, and my goal as a primary doctor is to proactively get patients the right care in the right place at the right time. If you do that, then it’s been shown, for example, that you can cut hospitalizations versus the status quo by 40% or more.

      Hanne: Incredible.

      Todd: It’s a significantly positive financial ROI transaction for them, because the way these arrangements work is that they get a share of the savings from keeping folks out of the hospital. That’s why primary doctors are so much happier when you put them in value-based paying arrangements, because under those arrangements, they can afford to spend that extra time with the patient. They can afford to hire personnel on their teams to help care for that patient, and give them the really straightforward care and support that those patients need to actually stay healthy [and] out of the hospital. 

      The primary care doctor, basically, generally speaking, doubles their pay under value-based payment by delivering care the way they always thought they were going to [when] they graduated from med school. As a bonus, then you significantly increase your pay, because you’re saving the healthcare system so much money.

      Vijay: Yeah. The proof of this is where you have a cohort at risk, and it’s the same provider, and the outcomes are fundamentally different.

      Todd: Oh, yeah. And the evidence is conclusive, as well as it being commonsensical.

      Hanne: It strikes me that — in the category of super, super obvious, but it’s a different way of measuring. It takes a different amount of time, a different perspective. So, how do you overcome that challenge, to measure how long somebody stays healthy for?

      Todd: One interesting challenge that primary care doctors face is the so-called “foot in two boats challenge.” It’s part of the exercise of transmogrifying from a fee-for-service paid operation to a value-based paid operation. Fundamentally, they’re two completely different kinds of operations. So, in a fee-for-service paid operation, if you’re a primary care doctor, you have to maximize throughput in order to actually stay alive. In a value-based payment paid operation, you’re focused on how do I take the best possible proactive care of my patients and keep them out of harm’s way.

      Hanne: Prevention.

      Todd: Exactly. It’s much more proactive, personalized care of people, and following up with them out of the office to ensure that they have their medications. Any changes in their circumstances actually get reflected in a change of their treatment. That things are going well, and they have the right support, and the right iterations are made to their care.

      Hanne: So, it’s a different muscle.

      Todd: Yeah, it’s a completely different mode of operation. And so, the “foot in two boats” is a famous articulation of this problem. What if part of my patients are being paid for fee-for-service, and part of my patient panel is being paid in a value-based payment mode? Then, I’m trying to do max throughput and also proactive, systematic care.

      Hanne: Yeah, and you’re stretched super thin.

      Todd: And you’re trying to be, like, two different modes at once. That’s why the most successful provider organizations under value-based payment have gone all in on one boat. And that’s helped by the fact that, for example, Kaiser and CareMore have a built-in health insurance plan, inside themselves, that then actually pays — in a value-based way — the providers that they employ, that operate in a value-based way.

      Scaling value-based care

      Hanne: So, is that part of the reason why we haven’t seen it happen faster? Because you have to create something from the ground up that is a total systemic shift?

      Todd: I think that’s a really, really good way of encapsulating why there hasn’t been this kind of shift nationally — because you have to go all in. As you think about the Kaisers, the CareMores, and even the ChenMeds — they take global risk payments from health plans, meaning, they go to health plans and say, “Pay me a global capitation payment, in effect, for all care expenses.” So they, in fact, are their own kind of mini-payer, if you will. Their existence proves that if you actually do that, and you get people the right care in the right place at the right time, it both leads to significant improvement outcomes and lower costs.

      The common denominator across the successful early American experiments is that they’re a full payer/payer provider stack who can therefore actually act with the right incentives. And, by the way, have the right information at the fingertips to then take the right care of people in the right place at the right time. That is a really tall order to replicate. The rest of America is neck deep in fee-for-service, and the payers pay a fee-for-service. The providers operate fee-for-service, and all of their business systems and operations are optimized for fee-for-service.

      Hanne: Yeah, it sounds like trying to renovate the foundation of a house, almost.

      Todd: Right. You’ve just got to honestly roll in with a new house.

      Hanne: Right. Build one.

      Todd: That’s right. I mean, that’s effectively the magnitude of the challenge. That’s a good way to think about it. And so, the question has been, “How do we, as a country, take those archetypes — take those results — and scale them to much more of the country — to the whole country?” As opposed to it being available and accessible to limited populations of people in certain pockets of the country.

      Vijay: The thing Todd really pointed directly to is that we have a sense for what to do, but how do you scale it? You have all this data and all this logistics. It’s just having to make tons of different decisions in complicated ways. This, ironically, seems like something that’s very well suited for tech, something where if you could build the infrastructure to do that, you could take all the little things that have to be done and do them that much more efficiently. Think about something like Amazon. Amazon and Sears, they both sell things. And actually, they both sold things over the internet. But, by taking a tech-first approach all the way through, whether we’re talking about the website, or the back end, or delivery, just everything — that’s the best bet to try to save and improve every little part. Because there’s not going to be a silver bullet that, like — with this one idea or killer algorithm, that suddenly healthcare is solved, or healthcare is easy or cheap. It’s going to be lots, and lots, and lots, and lots, of little things.

      Hanne: When you say tech from A to Z, and there’s no one silver bullet, but it’s all little aspects and incremental accelerations or improvements, what exactly do you mean? Where does it look different?

      Todd: You have to think about it as honestly reinventing each layer of the stack of American healthcare. You want to have a health insurance plan layer that’s explicitly optimized, from birth, to support value-based payment of care. So, the historical American health insurance company was born in a world, as Vijay said, where they’re paying fee-for-service bills. And so, that’s what they did. As those bills began piling up, escalating speed, their response was, “Okay, I’m going to erect administrative infrastructure that micromanages doctors and patients, and polices what they’re able to do, by making them ask me and give permission.” That’s probably not how they pitched it to people, but that’s effectively what it was.

      Hanne: Yeah.

      Vijay: Because the one thing they can do is say no.

      Todd: Right. So utilization management and pre-authorization, right? And, things that, for all the physicians listening to this, are epithets. That, then, led to doctors erecting their own administrative infrastructure to interact with the payers, to make arguments about what should actually be paid for, which has massively escalated administrative spending on healthcare in America. It also led to escalating mistrust, distrust, between patients and insurers, and doctors and insurers. It has generally not solved the problem. You want a payer that says, “Look, we’re not going to pay a fee-for-service, we’re going to actually pay in a value-based way,” which is a totally different mode of payment.

      Hanne: But that feels like it’s about, again, a kind of framework and mindset shift and payment shift. I don’t understand tech’s element in that.

      Todd: You cannot do anything I just said without software, and you certainly can’t do it scalably without software. So, first of all, the notion of paying a primary care doctor in a value-based way — you’ve got to be able to, for that patient panel, for that doctor, set the right risk adjusted global budget. You’ve got to actually be able to actually track everything that actually happens. So you’ve given the doctor visibility into what’s happening to their patient base. And you’ve got to provide the doctor with data from all points of the compass — pharmacy data, lab data, electronic medical record data, claims data about what kind of care that patient’s getting from across the system. To put all that at the doctor’s fingertips, to actually make sure this person is healthy. Beyond that, you need to establish a relationship with a member that is incredibly supportive, where you’re also getting the member information about where they stand and what needs to happen, before they even know it.

      Hanne: So, there’s like a whole other information flow happening.

      Todd: That’s right. Mark Smith, who’s a visionary healthcare leader, has this great analogy. He says, “I go to Harrah’s casino. And before I even know I’m thirsty, there’s someone there with a drink. Just when I’m about to leave the blackjack table, someone says, ‘Can we comp you some free food? Take some chips while you’re at it.’”

      Hanne: As if by magic, yeah.

      Todd: Exactly, right. And so, he says, in all seriousness, healthcare needs to be like that. So, before you even realize that you’re about to have a problem, someone calls you and says, “Hey, I think you might want to actually get this med. I think I want to see you and check something out.” If the American health system operated like Harrah’s casino, it would save many more lives and cost a lot less. 

      This is a classic data and tech problem. I have friends who work in AI in healthcare, and they said, “Look, at this point, the holes in American healthcare are so big you can see them from space.” So, if someone hasn’t refilled their med — they have a chronic illness, they don’t take their meds, they’re going to the hospital. And so, the low hanging fruit in U.S. healthcare is so plentiful — it’s fruit pies on the ground, right, with a cold glass of milk next to it. There’s so much progress we can make if we apply tech-enabled process control and improvement, as has been routine in virtually every other industry, to healthcare. But to do that, you need to actually have a full stack, payer provider ecosystem, where there is a business case for the use of those approaches.

      Vijay: All of the innovation we’ve been thinking about has been — let’s say, in therapeutics or how medicine is done — what we’re really talking about is basically how to get better at the existing game. If the game is fee-for-service — how to do lots of services, better services, more services. Almost like if you’re building a machine to do chess, you can do chess really fast, but this is about getting rid of the chessboard entirely and playing a different game. The real point is in giving us the whole stack. That’s really, really hard to do. 

      But to your point, I think it is particularly intriguing to ask now, if we’re going to start with a new game, how can we set up the game to have the best chance to win — the best chance to benefit the patients, and to reduce costs? How do you construct that game? How do we make sure that we’re constructing the right game? Because whatever game we do, someone’s going to try to win, and will win at those rules. But that may not be what’s best for patients and may not be what’s best for cost.

      Hanne: It’s interesting, because when you’re describing this, I’m thinking about all the ways that healthcare as an industry is particularly challenging to introduce innovation into, because of how unique it is, and certain regulatory hurdles. And I’m also thinking about the move towards more consumer facing, and the market-driven forces that are pushing in that direction. So, I’m going to ask either a really hard or really dumb question. I don’t know which it is, but — how does this value-based shift work with that, kind of, market-driven shift? You know, you see these two big forces. Are they opposing tides? Do they come together? What’s the fit between those two?

      Todd: So, in a nutshell, look — if you lower costs and improve outcomes, you can actually put a better health plan product in front of people in a market like the Medicare Advantage market. And more people will buy it.

      Hanne: So it’s quality really?

      Todd: It’s quality, and it’s cost savings. It’s then the ability to use those cost savings to fund better benefits in your health plan than the competition. And this intersects with consumerization, right. So, in a market where you can offer a health plan that is better for consumers, there’s also a significant positive ROI in investing in and delivering on a world class consumer experience. Why is this? Because if you actually deliver a world class consumer experience, that’s both helpful to you as you seek to win more customers, but it’s also directly helpful to your ability to deliver better outcomes and lower cost. Why? Because if you’re actually working with a member, and they do not trust you, then when you ping them and say, “I think you…”

      Hanne: It all breaks down.

      Todd: Yeah, the member will say, “Pound sand. I don’t trust you further than I can throw you.”

      Hanne: Yeah. So, it’s about strengthening the relationship so that when you need to direct them, it works.

      Todd: Exactly. Both when you need to direct them, they will actually listen to you, and B, when they have a problem, they’ll call you.

      Hanne: Oh, wow.

      Todd: As Atul Gawande says, “The true superpower of a primary care doctor is that people will tell them things that they don’t tell anyone else.”

      Hanne: Oh, my gosh.

      Todd: So, basically, a patient will call a primary care doctor and say, “I am not feeling so great about X,” right? And that’s an early warning signal to jump on. So, if you are a health plan, and you are genuinely trusted by your members, not only will they listen to you when you call them and suggest something — or ping them and suggest something…

      Hanne: They tell you more important stuff.

      Todd: They will call and say, “You know, I’m having this issue. I’m not sure what’s going on. Can you help me?”

      How to shift towards value-based care

      Hanne: Yeah. So, if this shift towards this model depends on gathering new types of data and knitting them into a more holistic picture of the patient for the provider and for the whole healthcare system, what are the types of information that we’re using now that we haven’t been using before? How are we thinking about that whole picture of the patient in a different way? And what is it that we need from the provider lens for this whole model to work?

      Todd: Yes. So, building on what we were talking about earlier about the operation of a primary care physician practice, you’re really moving from a paradigm where you’re maximizing throughput to  a paradigm where you are maximizing the outcomes you’re delivering to patients. And so, armed with the right comprehensive data, you want to actually make sure that you are getting members the right care in the right place at the right time in a highly coordinated, proactive way. The role that technology plays in that is not just the assembly of data and the catalyzing of the right action, but also actually enabling virtual care and home care on an epic scale. Because it’s increasingly obvious that the right place for the right care is the home.

      Hanne: Right. Not getting you into the ER, not in a hospital.

      Todd: Or forcing you to come to the medical office, right. So, going to the early examples of someone who has diabetes, or hypertension, or congestive heart failure, as opposed to you going into the medical office, which is a big logistical exercise. Getting a measurement taken, and then the doctor deciding to put you on this particular med and saying, “Come back in three months.” You come back in three months, they make another adjustment, and then so on and so forth. That takes months or years to get your chronic condition under control. 

      Instead, a patient has a continuous glucose monitor that streams data in the software. It has a wireless scale — streaming data into software. It has a wireless blood pressure cuff — streaming data in the software, which then analyzes the data in combination with humans, if necessary — then triggers a set of actions where the care provider can, basically, through a televisit, say, “Okay, I’m going to adjust your med by X,” and then see in 24 hours what the result was. And then adjust it again, then adjust it again, and adjust it again. Within a matter of one to two weeks, literally get that chronic condition under control — vis-a-vis a pattern in the old world, where it would take years, and in a way that’s dramatically more convenient for the patient. You’re basically dramatically — through remote monitoring and virtual visits aided by software, you’re both detecting issues a lot earlier, and you are increasing, by orders of magnitude, the speed of “intervene, see what happens, and adjust.”

      Hanne: The whole feedback loop becomes much faster.

      Todd: Exactly. That’s one of the many possible examples. But overall, that’s what’s happening. The accessibility of care dramatically improves, the richness and the timeliness of information dramatically improves, the frequency of touch points dramatically improves. And you experience a significant acceleration of improvement of outcomes, and associated with that, lowering of cost.

      Hanne: So, I’m thinking about providers listening to this and thinking, “Well, the information itself sounds like a dream, to have all that at your fingertips, to understand all that about your patients. But actually parsing all of that?” How do we make sure that we deliver that to providers in a way that it’s not this giant, hairball mess of more data and information that they then need another administrative layer just to figure out?

      Todd: Well, this is a classic problem where software could help — is helping today. What I would say is that the optimal approach to acting on this opportunity is where you’re fusing software and humans together in an optimal combination, such that you could actually deliver the actual result.

      Hanne: Right. So, it’s precisely where those two meet that you have to make sure it’s joining well.

      Todd: Yeah. You want to design a tech-enabled service, which leverages both software and humans in the right combination with software doing a ton of the work to be able to efficiently, effectively and scalably deliver the actual outcome to the patient.

      Hanne: So, if we’re talking about this massive shift from treating sick people — treating [the] chronically ill — to keeping people healthy, healthier. Preventing illness, catching it earlier — how do we begin to shift the whole system conceptually around that? What other types of information do we need to be thinking about? Transportation, food security? Or new types of treatments that we haven’t been thinking about?

      Todd: If you have a full “payvider” stack — and by that I mean a payer and provider healthcare stack that is optimized for value-based payment and care. One inevitable additional layer of action you take on is non-clinical drivers of terrible health outcomes. And again, you talk to every primary care doctor in America, they say, “A huge portion of what actually drives terrible health outcomes for my patients are not clinical.” So, social determinants like transportation, food, how your house is equipped, social and emotional support, etc. I’ve talked to primary doctors in Florida who say, you know what the number one thing that you could do to help people with COPD in Florida, and reduce adverse clinical events?

      Hanne: No. What?

      Todd: Get them air conditioning.

      Hanne: No.

      Todd: One of the things — it makes a lot of sense for payvider stacks to provide aid to members that includes fixing up your bathroom, with bars and mats that stop you from slipping and break your hip.

      Hanne: Oh, my gosh, yeah.

      Todd: To be able to actually get you healthy food that is tailored for your chronic condition, and other non-clinical artifacts and services that have a huge impact on ultimate clinical status and health status.

      Hanne: It’s so interesting, because while you’re telling that story, I’m thinking about the unseen caregivers of those patients that are doing those things now, and how if the healthcare system can take over that role of the son, or the daughter, whoever it is taking care of that person that needs that extra air conditioner, or needs the handrails set up. It feels like the healthcare system is doing more, but it’s actually making it more cost efficient. It feels like everybody wins then.

      Todd: I think you hit upon something very important there. In that, for the average patient, the American healthcare system can seem very forbidding, and confusing, and fragmented. And it’s very hard to understand. It’s the healthcare system that only works when it works, because the individual doctors and nurses in it are by far the best in human history. The problem with our healthcare system isn’t the doctors and the nurses. Far from it. It’s that the system in which these professionals are operating is so disorganized, fragmented, confusing, incentive misaligned, information poor, reactive. The notion of having a professional daughter and son to help you navigate the system as a patient, and be able to get the right care, right place, right time — is, I think, a crucial, crucial role. That’s, again, another role where technology can be extraordinarily helpful in helping that happen. So, a major barrier for a lot of patients is literally transportation. This ability to literally get to your doctor’s appointment, or get to where you’ve got to go as you travel through the healthcare system. And so, a very significant benefit to patients getting the right care, right place, right time is to actually provide them with medical transportation.

      Hanne: Right. It’s not just the doctor’s visit, it’s getting to and from the doctor’s visit.

      Todd: It’s literally getting to and from the doctor. Key. Really important, completely solvable problem in a tech-enabled way.

      Vijay: You know, the thing about social determinants is that I think there’s so many more low lying fruit. It’s not something where people have put much effort. It’s not something that payers have really thought about. 

      I think there’s a huge opportunity for just actually applying analytics, data science, tech, to figuring out what these low lying fruit areas are, and what could be done about it. It’s probably not even a hell of a lot of dollars, it’s probably just figuring out what’s the best place and what’s the biggest need, and when is that need. And that combination of who, what, where, when — that’s particularly hard to figure out. If we could provide a coach or a parent, or, like, a doctor that was with you 24/7, that person would probably know, but we can’t know that. So, that is where tech can sort of fill in the gaps.

      A new future for healthcare

      Hanne: So, just to go back to where we began, and think a little bit about this major new orientation towards, like — it’s a very obvious shift, but it’s a massive shift to make for the system, right? When the entire architecture of the system and the way it’s been developed over the last decades, and the mindsets and the education and the processes — all those things have to be shifted. If you were able to full stack the entire system — if you were able to start from scratch — what would that look like today? What does a real revolution American healthcare look like?

      Vijay: I would love to just go right at the question, which is, what would it take to make the American healthcare system the envy of the world?

      Todd: I think a way that the U.S. can leapfrog, going from the bottom of the rankings in the developed world to the top, is to make an increasingly strong move toward value-based payment care. See the rise of more and more tech-enabled payvider stacks across the country. Have those tech-enabled payvider stacks, in all their different forms, compete with increasing energy on the basis of outcomes and cost and consumer experience.

      Vijay: And fueling innovation that way.

      Todd: And fueling massive innovation, right, versus if the entire U.S. health system were actually being run centrally. We just have to get to a place where we create the right magnetic field from an incentive standpoint, by continuing to move strongly toward value-based payment. So that value-based care innovation and value delivery innovation can really blossom through these tech-enabled payvider stacks and all their configurations competing with each other.

      Vijay: Well, how can that come to be? I think that makes a lot of sense from, sort of, a bottom up, but how do we get to it from the top down?

      Todd: I actually think that in the Medicare space, all the conditions already exist to enable this to happen. It is entirely possible to build tech-enabled payvider stacks that are wired for value-based payment and care with no incremental policy change. One key to that is that in the Medicare space — say the average Medicare Advantage plan keeps a member for eight years. The cycle time between intervening and helping and saving money is short, given the population. And so, that means that if you’re a tech-enabled payvider, or tech-enabled payvider stack in the Medicare space, you have a very strong business case to make investments in better care. And you’ll realize the payoff. One challenge in value-based care oriented payvider stacks in the under-65 segment of the population is that people tend to have their insurance plan for far shorter than eight years. And so…

      Hanne: And that’s because you change jobs, you change…

      Todd: People change jobs.

      Hanne: …situations. Yeah.

      Todd: They go from Medicaid, to the exchange, to an employee and back. And the cycle times between intervention and payoff, on average, tend to be longer, because you are dealing with a population that doesn’t have the level of illness burden of seniors. So, I think a really interesting area of policy innovation could be how would you actually solve that problem? Because if you did, then I think that it would then create the right kind of business case support to do the tech-enabled payvider play outside the Medicare space. I think it’s still possible to do, it’s just a lot harder for this reason.

      Hanne: And to your point about the importance of the relationship with the patient being such a powerful tool when you only have a short-term relationship.

      Todd: That’s right.

      Vijay: You know, what we’ve been talking about trying to innovate a complicated system of analytics, decision making, logistics. These are all things that are, in many other areas, well approached by tech, well improved by tech. If you think about even just the A/B testing for websites, or for services, or for anything — that just constantly, sort of, trying new things, experimenting, having the analytics, seeing if it improves, and carry on. Ironically, that’s also not alien to medicine, that’s also an RCT, in a sense. I think we need to innovate, we need to try new things, see what works, see how it changes outcomes, and incent people to do it. Once you realize that you can set up a system where we are incented for innovation, that will automatically bring tech in.

      Hanne: A system that’s incented for value, that’s incented for innovation, and that’s incented for actual health before we even get sick.

      Thanks so much for joining us on “Bio Eats World.” If you’d like to hear more about all the ways biology is technology please go subscribe to the a16z bio newsletter at a16z.com/newsletter, and of course, subscribe to “Bio Eats World” anywhere you listen to podcasts.

      • Todd Park

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      • Hanne Winarsky

      Amazon Narratives — Memos, Working Backwards From Release, More

      Colin Bryar, Bill Carr, and Sonal Chokshi

      When you hear stories about Amazon’s “invention machine” — which led to a company with not just one or two products but several successful diverse lines of business — we often hear about things like: Memos, six pages exactly and no powerpoints at all!; or, the idea of just “work backwards from the press release”; and other such “best practices”… But what’s often lost in hearing about these is the context and the details behind them — the what, the how (as well as their origin stories) — not to mention how they all fit together. Knowing this can give us insight into  how all companies and leaders, not just Amazon and Bezos, can define their cultures and ways especially as they scale. After all, Amazon was once a small startup, too.

      So in this a16z Podcast with Sonal Chokshi — the very first podcast for the new book Working Backwards: Insights, Stories, and Secrets from Inside Amazon (out February 9) — authors Colin Bryar and Bill Carr share not only how Amazon did it, but how other companies can do it, too, drawing on their combined 27 years of firsthand observations and experiences from being in “the room” where it happens. Bill was vice president of digital media, founded and led Amazon Music, Amazon Video, Amazon Studios; and Colin started out in the software group, was a technical vice president, and then, notably, was one of Jeff Bezos’ earliest shadows — the shadow before him was in fact Andy Jassy, president and CEO of Amazon Web Services (soon to be CEO of Amazon).

      The two share not only the early inside stories behind (ultimately) big business moves like AWS, Kindle, Prime — but more importantly, the leadership principles, decision making practices, AND operational processes that got them there. Because “working backwards” is much, much more than being obsessed with your customers, or having company values like “are right a lot”, “insist on the highest standards”, “think big”, “bias for action”, and more. The discussion also touches on hot-topic debates like to lean-MVP-or-not-to-be; the internal API economy; do you even need a chief product officer; and if you need less, not more, coordination as you grow. Can startups really be like Amazon? Yes: and it comes down to how leaders, organizations, and people at all levels decide, build, invent… using the power of narratives and more.

      Show Notes

      • The early days of Amazon and the development of the company’s 14 principles [3:18], including one that says leaders should be “right a lot” [6:53]
      • Amazon’s use of written narratives over PowerPoints [9:05], and how meetings are conducted using narratives [15:55]
      • The tenets that guide Amazon’s decision-making [24:00]
      • Working backwards from a press release [26:28], using AWS as a case study [34:56]
      • Discussion of lean startup principles [37:31], and how Amazon’s core principles balance each other [46:21]
      • Advice for startups, including reducing coordination and centralization [51:16]
      • Lessons learned from shadowing Jeff Bezos [58:50]

      Transcript

      Sonal: Hi, everyone. Welcome to the a16z Podcast. I’m Sonal, and today I have another one of our special, exclusive first-looks-at-a new-book episode — and it is both a very timely and evergreen topic, because the new book, coming out this week, is titled “Working Backwards: Insights, Stories, and Secrets from Inside Amazon.”

      In it, authors Colin Bryar and Bill Carr — who between them have a combined 27 years of experience in the company — where Bill was vice president of Digital Media, founded and led Amazon Music, Amazon Video, Amazon Studios for a decade. And where Colin started out in the software group, was a technical vice president, and then notably, was one of Jeff Bezos’ earliest shadows, a legendary program there. Fun fact: the first shadow before that, I believe, was Andy Jassy, president and CEO of Amazon Web Services (and now to be CEO of all of Amazon). The book actually shares the origin story of AWS, among other businesses there, which we touch on briefly — though, as a reminder, none of the following is investment advice. Be sure to see a16z.com/disclosures for more information.

      But in any case, our focus today is really on what is the Amazon way — and can other companies really adopt certain best practices, too? In fact, as fast-growing companies establish and find their way, how do they define and scale their culture, processes, and more? We actually spend most of the episode digging, in detail, into two operational practices in particular — the infamous memos-instead-of-PowerPoints, and working backwards from a press release and FAQs. Given the presence of those two topics in tech folklore, and lots of misunderstandings as well — so I actually probe for the origin stories, the specific details of how they do and don’t work, and other nuances so organizations of all kinds can take what they want or need.

      Finally, we also debate within this episode the tradeoffs of lean and minimum vs. maximum viable products (and whether the emphasis is on the wrong letter there); whether product managers have a role in companies organized like this; and more topics throughout.

      Amazon’s early days

      But we start by very briefly discussing the foundational principles — and actually, the first question I want to start with, especially since I hate the “why’d you write this book” question) — Colin, Bill, honestly, there’s a tendency for folks telling these stories, these kind of narratives, to do a sort of hindsight is 20/20, not accounting for attrition data or the failure cases as well. So, part of me is skeptical that startups can do what Amazon did. And, what’s most notable, too, is that Amazon had not just one or two or three product lines, but literally entirely different yet successful lines of business under one roof. So, what drives that? And can startups really relate to the Amazon story then?

      Colin: Well, one thing is that the businesses you mentioned, they are substantially different. They require different expertise, they’re different customer sets — AWS is B2B, there’s streaming video, there’s the e-commerce business — but they all have one thing in common, and that’s what we talk about in the book, and we call it The Invention Machine. Which was the process and principles that Amazon used to develop these businesses. And the ones that we talk about, they started off as ideas on a whiteboard or emails — which many people were skeptical we should even do. Some of them grew into household names, but they all started off very small, and with just one or two people.

      Bill: I would first start by going back in time a little bit, and place you sort of where Jeff Bezos was and where all entrepreneurs start out. So, at the beginning, Jeff worked out of a simple office building with a handful of employees, and he would be hands-on for everything. The very first customer support emails, like, Jeff wrote or co-wrote. He could review the work and think about all the policies. He could direct the team, and set the tone and the pace.

      Well, that works just fine when you’re an early-stage company and you know there’s fewer than 20 of you, and you can all do a stand-up each morning, and you can be hands-on. But guess what? That completely breaks once you start to grow like a weed, and you found product-market fit, and suddenly you’ve looked around and realize you’ve got a team of 130, 200, 500 — and, you realize that there’s all kinds of decisions and meetings happening around you. You can’t be part of every decision. And so, to me what’s most remarkable and notable, is that Jeff sought to figure out ways to actually inject his lens of thinking into all those meetings — and then sought to create a bunch of processes that would reinforce the way he would think about the work, or do the work itself.

      Sonal: It’s this idea of operationalizing ‘the invention machine,’ as you guys are describing it. Some of those principles and processes — at a high level, to quickly summarize, to set context for our listeners — they range from customer obsession, ownership, invent, and simplify, “are right a lot” for leaders, learn and be curious, hire and develop the best, insist on the highest standards, think big, bias for action, frugality, earn trust, dive deep, have backbone, disagree and commit, deliver results — which, I love. And we don’t have to unpack all of those, because I will take, like, all day, and it’s the whole reason your whole book exists.

      I do want to ask about one before we go into some of the other practices, which is around leading. And the one that really intrigued me was #4, “are right a lot.” And you basically write that, “Leaders are right a lot. They have strong judgment and good instincts. They seek diverse perspectives and work to disconfirm their beliefs.” And I love this, because I always think to myself, “Yeah, dammit, a leader should be right all the time. Their instinct should be damn good.” Tell me more about that one, because that one made me chuckle a bit.

      Bill: Yeah this is — and just to be clear of course, we did not write that. That is Amazon’s words that Jeff and the S-team, being his direct reports, painstakingly reviewed to come up with that exact language to define that principle — as with all 14 others. They sweated over the details of every word, every sentence.

      And, this principle, “right a lot” — in some ways, it’s very straightforward, which is that as you move up, an early-stage CEO as they grow and progress, they need to be more in the mode of delegating important work and auditing work, but their most important job frankly is to make decisions. And, many decisions will come to you — whether that’s presented with a spreadsheet, a document, PowerPoint. There’s all kinds of data that will be presented to you with those decisions, but there are very, very few problems where the data gives you the answer. At the end of the day you’re going to have to use your judgment.

      What “right a lot” refers to is, number one, when those leaders are confronted with those decisions, that more often than not, they pick the right door, but the second part of the definition refers to, more importantly, how those leaders make decisions. Which is, a lot of people think that leadership is about their very strong opinion, arguing their opinion, and winning that argument. And what great leaders do, actually, is they can stake out a position — but they are willing to update (mean change their mind) on what is the right answer, based on new information.

      And you know, even Steve Jobs talked about this at Apple where, some product that they launched, it ended up being a mistake. And it was — one of his reports had been telling him all along, “We shouldn’t do this, we shouldn’t do this, we shouldn’t do this.” And then after it launched and it failed, he came back to that person and said, “Why didn’t you talk me out of this?” And the guy said, “Wait a minute, I tried to talk to you out of it. What are you — what are you talking about?” And he said, “Well, you didn’t do a good enough job, because you didn’t present the evidence to me in a persuasive enough way to make me realize why your point of view was so important.” And so, it’s thinking about it and framing it that way — about bringing forward the right data, and the right information — and then it’s also the job of a leader to solicit that to make the best decision.

      Narratives over PowerPoints

      Sonal: Well that is a perfect segue to a question I’m dying to ask you guys about — because so many companies, their culture is like the mission statement and the values that you’ve shove in a drawer. And so people spend so much time talking about the words and the precision of what they want those principles or values to be, but not actually how to operationalize it.

      So, in that vein, because your book really does outline how to operationalize that through the processes and practices that you guys share. One of the ones that comes up all the time in Silicon Valley folklore is the infamous “no PowerPoints, write a memo.” Let’s tackle this one first, because what you just shared, Bill — about “convince me, share the evidence in a persuasive way” — the point is to be effective and be heard, you have to do that well. So, what is your best advice about how leaders and people in the group can share that information?

      Colin: So, Amazon started experimenting with writing narratives in 2004. And it was a result of weekly meetings, four hours every week with the S-team (Jeff’s direct reports), where 2-3 teams would come in and give either updates on their business — it could be a decision that needed to be made, or investigating new areas to go into. And the business was growing fast, and we realized that we were not making the right types of decisions. Some of the meetings would go over, we never really accomplished what we wanted to.

      I was Jeff’s TA [technical advisor] at the time. We had been looking at other ways to conduct meetings, and we were big fans of Edward Tufte, who’s professor emeritus at Yale, he came to Amazon to speak a couple of times — and after one particularly painful meeting — it was later on in the week, it was toward the end of the day — Jeff said, “Let’s stop doing PowerPoints at these S-team meetings. It’s the wrong tool for what we’re trying to do, and let’s switch over to narratives.”

      Which are really just now a six-page memo. One thing that is a little bit misunderstood is these ideas don’t come out fully formed. They started out as four-page memos — it ended up that six-page was about the right length for an hour meeting. But, we did it because narratives convey about 10 times as much information. You know, the pixel density is about seven to nine times the pixel density. People read faster than people talk, and you can have multi-causal arguments in a narrative much better than a hierarchical PowerPoint. But we realized we just needed a better way to analyze complex situations and make better decisions. So we just experimented with this. And the first ones were not good.

      Sonal: Just to quote you guys, because this actually made me literally laugh out loud. You guys, write, “The first few narratives were laughably poor when evaluated by today’s standards. Some teams ignored the length limit…” blah, blah, blah. And I was like, yes, people are not — I mean, we may be wired to be good storytellers, but writing is actually a hard skill. I do worry that it becomes a bit performative. For the best writers, not just the best presenters, because — one of my colleagues when I used to work at Xerox PARC used to call this “pissing on paper.” Which is, like, this idea of, you know, how dogs piss around their territory. There’s also a real-time component, where people are performing in real time, like, leaving comments in the middle of the meeting.

      Colin: So, a common misconception is, well, now it’s just the best writers instead of the best presenters that’s gonna win out in narratives. We haven’t found that to be the case. It’s really the best thinkers [who] write the best narratives. And some of the best narratives I’ve ever read are by people whose first language is not English.

      Bill: Yeah, in fact, the best narratives, many of them are written by software development engineers who may not have even focused on their writing skills. A good narrative, it’s a very data-based and fact-based document. And, writing good narratives is way harder than making a good PowerPoint. And I think, honestly, a lot of companies don’t do this just because it’s hard. When Colin and I brought this to other companies, what tends to happen mostly is that the author will vomit out about 25 pages of just sort of raw data, at you. Getting that person to then shape it and narrow it down to six pages of well thought out narrative is really hard. And oh, by the way, if you spend 10 hours a day reading detailed narratives, I’ve got to tell you, it can be mentally exhausting.

      But when people muse, like, how does Amazon do it? Like, how is it possible that they can effectively manage such diverse businesses? One of my number one arguments is that they use narratives to conduct meetings, not PowerPoint. And actually, a former colleague of mine (Derek Anderson, who is now the Chief Financial Officer at Snap), I think he made the observation once that Amazon has like a “narrative information multiplier.” It’s a strategic advantage that the company has over other companies, because <Sonal: love this> their executives are, like, 7-8 times better informed about what’s happening in their company, and they’re able to give super granular, specific feedback to those teams.

      Colin: It allowed the S-team to stay connected at a much deeper level. As Amazon started to move into more and more businesses, the span and control of the S-team didn’t grow by 100X — it’s still a relatively small team, and they are just as involved when it was a small company, as they are now when it’s a large company.

      The other thing I’d add about narratives is it does remove a lot of bias from the process, where you can have a charismatic speaker with a so-so or even a bad idea that convinces your company to do something that you should not do. You know, the converse is also true. If you have a shy engineer with a great idea, but it’s a boring presentation. It’s one way in which Amazon removes bias to make better decisions and the idea floats to the top rather than how good of a talker the person is.

      Bill: And we could go on about this forever but, the other part is it actually is a great way to get your team more engaged from top to bottom, especially in a COVID time. The way that these meetings work is you share the document — and you know, whether it’s with G-Suite or with Word — then everyone can then use the comment feature, and it doesn’t matter whether the person making the comment is a C-level person or a fresh-out-of-college individual contributor, all those comments get seen and heard.

      And then when you get into the discussion phase, all those people actually can then understand, you know, why we’re making the decisions we’re making. When you do a PowerPoint, you have to wait to get to the punchline, and while you’re waiting, you’re not sure, like, well, what are we actually doing in this meeting? With a narrative, you just take all that in, and so then you can have high-quality discussion versus that interruption and disjointed conversation you have with a PowerPoint. And if you missed the meeting, you can just read that document. So, there are so many ways in which the narrative method is superior to PowerPoint; that as you can tell, we can never go back.

      How narrative meetings work

      Sonal: Okay, so how does one do meetings then, based on these memos? One of the things you guys said is that sometimes it’s shocking, because the first 20 minutes of a meeting would be silent. And that made me chuckle, by the way, because one of the cofounders of Roam, the note-taking app, was sharing that sometimes they have entire meetings that are silent, because they’re just sharing notes with each other in Roam.

      Colin: Yeah, so Amazon meetings with narratives, they are strange to the uninitiated. It’ll be chit-chat before meeting when they were in-person or it’s online, people are starting to come in — but then there’s silence for about 20 minutes. And during those 20 minutes, people are just focused on reading. There’s no sound. They’re entering questions, and sometimes the presenting team can answer the quick questions right in the comments.

      So, for that 20 minutes, really there’s a huge transfer of information that you can’t see, and then the rest of the 40 minutes is really just high-quality Q&A and discussion on the questions that have already been entered or that come up. The six-page narrative is a forcing function to where you can cover that amount of information in a one-hour meeting.

      Sonal: Tell me a little bit more about what needed to change in the specifics of meetings, in terms of running them. Because the premise of this conversation is — not everyone starts out as Amazon, and that startups can do these things — I’m trying to really tease apart, like, are we just replicating the meeting dynamic in the memo and then the same thing happens anyway? People would begin reading in the meeting, but then how would the decisions happen? Like, what happens next?

      Bill: There are a variety of different kinds of documents you’d review in a meeting. It could be an annual operating plan, it could be a monthly business review, it could be a PR FAQ about a new product. So, the conversation, literally what you do once people stop reading is you just go page-by-page. Or alternatively, if it’s a smaller group, and let’s say there’s just, you know, 10 or 15, you could literally just go around the room and say, “Okay, Sonal you’re first. What questions and comments do you have on the document?” And so, you would get the feedback, questions, and comments from all participants. And then the document itself would present some sort of specific proposal. It’s asking for some budget amount. It’s asking for, are we gonna launch this new product? And, at the end of the meeting, you are tying it up and wrapping it up and saying either, “No, we agree that, what you’ve written in this document, that plan works. Approved, go ahead,” or you debate and discuss it.

      One of the things you might do at the end of the document is make a clear section after you’ve presented all the facts to say, you know, what are the decisions we need to make today? And then list those out. Or, what are the important parts of this plan where we need your feedback and input. Like, we’re not sure whether we should go down path A or path B. A good document will clarify what are the decisions we’re gonna make.

      Colin: The one thing I would add is that a lot of first-timers to narratives, right after people read, they say “Let me walk you through the narrative” and you stop that right away. You know, you’ve just had that 20 minutes of high-fidelity bandwidth — why dumb it down with a two-minute verbal walkthrough of the document? You wanna get to that feedback loop as quickly as possible, just talking about the document.

      Sonal: We didn’t actually really say what goes in the memo. You observed that the memos can vary in form and format by function, and purpose. But can you at least describe what goes in the memo specifically? Like, even the ingredients would help.

      Colin: Amazon has different types of memos for different purposes. So, if it’s some monthly, or quarterly, or annual review, there’s the typical “what were the key wins, what did we do wrong, and what can we do better” and “what are the key initiatives coming up for the next year.” You can have appendices, too, and the appendices are supplemental data that everyone’s not required to read in the narrative meeting, but if you do need to go jump to it to answer a question, you can.

      Bill: It would include tables, with, like, here’s a summary of our financial results from the prior year. Here’s the table — the summary of our plan for the next year.

      Colin: They also work very well with design, which you may not think about at first. But if you’re going over mockups — either UI mockups for an app, or physical prototypes of some hardware, or a process that you’re gonna build — having a narrative actually helps set the stage to say before we take a look at anything, here’s what we’re trying to accomplish with the user experience. Here are our goals, here are the challenges that we’re trying to solve. You know, I’ve been at mock-up meetings where everyone thinks they’re a UI expert, but if you don’t have that <Sonal: Yeah, oh god> before and the comments on, you know, three or four things where you should move this over here. But you don’t do that if everyone’s on the same page with reading a short narrative beforehand.

      Another thing that can be in a narrative that’s really powerful — especially for something that’s going on on an iterative basis, as if you’re refining an idea over time — are tenets. And those are really — okay, what are the design criteria that we are not gonna compromise on, or that we’re gonna fall back on when we have to make tough decisions. And that’s front and center, usually up in the beginning of the document. “Before we go over Amazon’s pricing policy, I want to remind you, here are the four tenets that we’re following to make the following decisions.”

      And you know, to get to those tenets, it was difficult. It took several meetings just to say these are the correct set. That’s a good caching mechanism, because if you’re a manager at a company that uses narratives, you’re gonna be context-switching and reviewing multiple ones every day. And sometimes you may only meet with the team once a quarter, and you wanna be able to very quickly get up to speed.

      Bill: There’s another important technique where you can actually add at the end of the document an FAQ section, for frequently asked questions. So, you’re actually anticipating the kinds of questions the audience are gonna ask you. And a talented senior leadership team is gonna ask you hard, probing questions about things like, “Well, you say in your plan that you’re gonna go do X, but it seems to me that there’s an important hurdle here of — you’re gonna need this important partnership, <Yup> you’re gonna — you have this important dependency. What gives you confidence that you’re gonna — actually can solve that problem?” And you would answer it. Not only does that help reflect whether the author and the team have good mastery of the issues, but it also helps speed up the discussion and the decision-making.

      A lot of my work leading — you know, Prime video was making very expensive multi-year agreements with motion-picture studios to acquire their films and TV shows for the service. These were big numbers, lot of money involved. If we’re gonna go spend that much money, we’re gonna go write up a deal memo that describes, like, okay, we think we should acquire these films and TV shows. Here’s what the package is worth, here are the detailed metrics that show why we think this is a good investment — or why not, in some cases, because sometimes we would review it and say we’ve looked at this deal and we’re not gonna do it and why. And again, just because it’s a Word document, doesn’t mean you can’t put in a chart, a graph, some independent input, an Excel table — all those things can still be in there, but then the narrative needs to clearly state with a clear beginning, middle, and end, like, here’s what we’re proposing to do, we’re not proposing to do, and why.

      Sonal: Yeah, I think that the thing I find most compelling is how much this mimics how writers think, obviously. One of the things that I found when I first came to a non-media company — and was working at one previously — was oh, my God, I have to make so many freaking PowerPoints, and I hate it, because it’s a muscle that’s not ideal for storytelling as much as people think it is. It’s not, it really isn’t.

      But then next, what I love about what you’re saying there is that the FAQs is actually the equivalent of doing the “inoculation technique” — which is what really good op-ed editors will really build in — and that’s not just because you’re trying to inoculate the counterparty to the arguments they’re gonna make — it actually, what you’re really saying there, and I wanna pull this thread, is — you now can have a better, deeper discussion, because you’ve laid to rest all the common things that you can literally get out of the way in, like, a memo in 20 minutes.

      Bill: Yeah, I mean, in a good document, that all comes out.

      Amazon’s guiding tenets

      Sonal: I wanna actually go back to the tenets for a quick second, and then pick up on a thread that you brought up. One of the things, Colin — you described it as a caching mechanism for the leaders, where they — you have to do a lot of context-switching so they get to kind of revisit that cache when they have to get back into that context.

      Colin: Yes.

      But I was thinking about it from the point of view of the group in the room, not just the leader. And how it’s — really sets the shared context for the room, to have the most collaborative mindset possible, while disagreeing within that framework. And in your book, you write that “tenets give the reader an anchor point from which to evaluate the rest” — but here’s the best part. “If the tenet itself is in dispute, it’s easier to address that directly rather than take on all the logical steps that derive from that position.”

      And sometimes you guys spent meeting after meeting just debating to get the tenets right — which I think is really great, because it allows you to actually separate the thing that the tenet is, and then what flows from that.

      Bill: Yes, I spent 15 years at Amazon, and I’ve since gone on — and one of the things that shocked me was that certain topics get, like, recycled for debate constantly, right? <Sonal: Yes. Yes, exactly> And I was like, whoa, we don’t do that. We didn’t do that back at Amazon. Well, why is that? And one is the use of a tenet, which is that all that debate and discussion can land on a fundamental issue about, like, what this company should or should not do. And if you don’t come to a common agreement on it, then you will be constantly — you’ll waste so much time with relitigating these issues.

      The second reason is that what the narrative process forces you to do is by actually putting down on a piece of paper, not only the tenet, but then, like, so here’s the specific plan — then, you set the date for the meeting and everyone reads it, and then you decide. And, I realized that a lot of those things get relitigated. So, it was one of the ways [in] which Amazon was very effective, which people don’t realize, as a management company.

      Sonal: Yep. On that note, does it then serve an archiving function for new hires and onboarding, that you scale that tacit knowledge that’s been made explicit? How does that work?

      Bill: Yeah, great point, because the other thing I also saw was whenever you hired someone new in — within a matter of a week, they would want to go relitigate. They too, <Sonal: Yeah, totally, it’s exhausting> would trip across, well why don’t we do blank? Or why don’t we — and it’s like, oh my gosh — and you can just say, “Look, here’s the tenet, here’s the document. You know, take a look at this, and then come back and, you know, talk to me.”

      Starting with the press release

      Sonal: Right. And then when you do decide to relitigate, it’s an actual intent <right> versus an accidental, every-new-hire-repeating-recycling <Bill: Right> the conversation. Okay. So the other big Silicon Valley folklore when people talk about best practices from Amazon is this idea of working backwards from the press release. Now, I know that you guys talk about “working backwards” well beyond — it’s the reason it’s the title of your book, obviously — but I really want to probe specifically into the mechanisms of what that means. Because, like, how does that happen? Is it just, “Oh, I wrote a press release for this thing I wanna do” — it’s a product, it’s an idea, it’s a service. I really would love to hear what, where, how, and why. And also, I’d love to hear the origin stories, if you have any specifics there as well.

      Colin: At its heart, the working backwards process is really starting from the customer perspective — and everything you do works backwards from that. The PR/FAQ (the Press Release and FAQ), that is the tool that Amazon uses to achieve the perspective of starting from the customer. It’s different than how a lot of companies develop ideas and products. A lot of companies use a “skills forward” approach. What are we good at? What are our core competencies? What are our competitors doing? How do we nudge into this adjacent market? How much market share can we get?

      Bill: When I was in business school, and taught to think about, like, how you expand and grow, as Colin already described — you create a SWOT analysis.

      Sonal: What is it, the strengths, weaknesses, opportunities, threats, right? Right.

      Bill: Right. But there’s no “C”, there was no customer.

      Sonal: Right.

      Colin: Amazon has put in deliberate mechanisms to make sure that the customer is front and center from the very first iteration of an idea. So, if anyone says — raises their hand and says, “I have a great idea,” the first thing that the manager or the person in the group will say, “That’s great. Why don’t you go write a working-backwards document.” And what that is, it’s two things: it’s a press release, and then a frequently asked questions document. And the press release has to clearly explain to the customer what it is you’re building — what’s the problem you’re trying to solve for the customer, and why does it make their life better? It can be something very small, or you know, it could be moving into a brand new industry. It’s a fractal process, which is great.

      Another thing is, these PR FAQ documents — it’s an iterative process. First of all, most of the ideas that go through it don’t make it out on the other side. And second is that it takes several, several iterations and feedback to refine before the project gets green-lit.

      Bill: In fact, the origin story of, like, how we got to the PR FAQ — or at least a part of it — both Colin and I were present for this, because in 2004, I landed on a new role working as one of the founding members of the digital media team at Amazon. For perspective, at the end of 2003, 77% of all of Amazon’s worldwide revenue was media products. But it was all physical media products. It was books, CDs, DVDs, VHS tapes. And, the writing was on the wall that, like, this business is not here to last. That — there were already a couple of million iPods that were sold. Millions of people were using Napster to file-share.

      Sonal: Napster, right.

      Bill: Right? It’s pretty clear that, like, okay — now that we have the internet, it’s just a matter of time before people, you know, consume their media digitally. But we didn’t know how. So, what I did — pulling out my bag of tricks from business school — is [I] marched into meetings with Jeff where, like, here’s our projections for how big the e-book business will be over the next 10 years. And here’s our projected market share. And, here’s what the pro forma P&L looks like. And here’s the kind of deals we’ll make with publishers. And here’s the competitive landscape.

      And, I was so proud of all this work, and he looks up at me and says, “Bill, where are the mock-ups?” And I didn’t have any mock-ups at that point. I was doing, you know, like, “Oh, here’s the projection — I’ll get started and we’ll start working on launching, you know, a better e-book store.” And, by “Where are the mockups?” what he meant was, this is all super interesting — or not really actually, is what he was saying — but what’s more interesting is like what is gonna be the customer experience?

      And more to the point, as we started to debate and discuss different ideas — whether it be in digital music or e-books — the discussion was about, well, why would we bother building, like, a me-too versus what you know Apple’s already got the iPod and iTunes. So what’s in it for the customer to have just have another — a knockoff of that service? Like, what can we build that actually creates real value for customers, something new we should invent on their behalf?

      And we tried mockups for a little while, but frankly there were, like, so many questions and so many details that we hadn’t thought out, and Jeff one day said, “Okay, I got a better idea. Why don’t we — everyone in this room — write up the idea for what they think we should go build in digital? We’ll write those things up, and we’ll come back in a few days, and we’ll read those, and we’ll go from there.” And this is before we’d done narratives. We had not, you know, figured out this PR FAQ concept, yet. But once we did that, everything changed. And suddenly we were reading, you know, one document was describing, like, a puck that would sit on your countertop, and you can talk to the puck, and could order groceries from the puck. I described the document about, you know, what we might go do in digital music. Another one was describing an early version of what Kindle might become.

      But some of these documents were, like, eight pages long. We needed to get this to be, you know, more pithy and clear. And Jeff said, “I know. Let’s write the press release for each one of these ideas instead.” <Sonal: Ooh, neat> And he said, you know, we should read the press release, because normally that comes last. And he said, you know, normally the engineering group and the product group, they go and they come up with the idea for the product, and then at the very end, when it’s time to sell it, they chuck it over the wall to the marketing team and say, “Okay, figure out how you go sell this thing.”

      And he said, but what if by the time that thing got to the marketing team, they said, “Yeah, well, the thing you built — for that really to work, we actually need that to have a price point of $99. But you’ve built it in such a way that it’s costing us $150 to manufacture each one — so, yeah, we’re not gonna be able to sell too many.” In other words, if you had known upfront that you needed to hit a BOM [bill of materials] of less than $99 to make this product go, then you would have designed the whole thing very differently and understood the constraints. It might have a whole lot of — there might be vaporware concepts. There might be concepts that you don’t know how you’re gonna solve. Maybe business model problems — but then in the FAQ, then that defines, okay, what are the hard problems we’re gonna have to go tackle to make this exciting product a reality?

      So, we switched to that method, and it was halting progress, and had we not really taken that approach, we would have not ended up with what turned out to be a breakthrough product, which was the Kindle.

      Sonal: That’s fantastic. So, a couple of questions to just quickly probe on a few nuances: First of all, I know what you’re really saying is, flip the perspective from inward-out to outward-in — which I think is really interesting, because that’s something I constantly think about content. Like, orient it in the value to the reader. But what would you say on the flip side, to the crowd that often says that part of the problem with “working backwards” [is], “Well, then you’re not really inventing what they don’t know what they want.” And how does one write a press release for the startups out there thinking, “Oh, well, you know, we’re creating a new category — this is not something that exists.” Tell me more about how you might address that crowd with this PR FAQ approach.

      Bill: I would submit that, in fact, that’s what this process is actually designed for. In 2004, when we started on digital media, the way e-books were, you could only read them on your PC. There were no, like, offline readers and tablets and devices in those days. They were priced way too high — like, the same price as the hardcover book. The e-books had existed for four or five years before the Kindle launched in 2007. But it was a tiny, tiny business. And it was a tiny business, because no one had imagined, well, what do I need to create, what’s the new thing I need to build to make e-books work?

      We defined what would be the ideal reading device — everything from, you would be always connected to the internet (which by the way, devices weren’t that way back in 2007). That you wouldn’t have to create some separate account or link your account to some mobile carrier. That when you got it out of the box, it already knew who you were, and so if you had actually bought a bunch of e-books online, they were magically already uploaded onto your device. All those kinds of things would be described in the FAQ section to say, “Yes, here’s hard problem one, and here’s how we’re gonna solve this problem” — or, “we don’t know how we’re gonna solve this problem yet, but here’s our path for how we’re gonna go work on that.”

      Colin: Yeah, I would just say, there are two very real use cases where Amazon used the working backwards-process to create something from scratch. With AWS, you know, cloud computing didn’t exist. And as we’re working through the Kindle issues — you talk about context-switching — an hour later we’d go to another conference room, and we’d be with Andy Jassy (who’s the CEO of Web Services), and we’d be reviewing Word documents about what would eventually become cloud computing.

      And it took us about two years to come up with, what are we actually trying to create? And you know, the first two were centered around storage and compute, eventually. And the press release — one area where it really helped to crystallize everyone’s thinking is — we came up with the saying that we want people in a college dorm room to have the same access as an Amazon developer to world-class infrastructure. And that <Sonal: Wow…> was just a powerful metaphor about, okay, so what are we creating — and, you know, what is this infrastructure that we really need?

      And starting from the customer experience, we had many, many small teams throughout the company just screaming at Jeff, at the infrastructure team, “I built my service, it’s taken me too long to deploy it and get it out and ready for customers.” And that’s where it started off as provisioning, but it kind of morphed through this working-backwards process to compute. And then in terms of storage — there’s a whole bunch of different types of storage. It narrowed down to simple storage service, was the very first thing, and then we would build out from that.

      Sonal: I love that example, because it really emphasizes this point, that when you start with the PR FAQ, you’re essentially starting with the differentiation. Because you’re really thinking in terms of the value — because there’s a million options for people to pick from — so that’s when you go from provision to compute.

      Because you’re thinking, if the customer is this kid in the dorm room having access to — and we often talk about the power. Like, actually Marc Andreessen, in his original 2011 “Why Software is Eating the World” op-ed, points to the power of this movement — like, you know AWS has been huge in bringing — we even have this op-ed about why every company is a fintech company. We actually call it “the AWS moment in fintech” — it’s quite amazing the impact that AWS has had on the industry, and there’s no question about that. But that precise point — of starting with the PR FAQ — to take what could have just been like, “Oh, here’s some storage, and here’s how to provision the services you need,” to here is how to create an entirely new business, that’s a whole different game.

      Lean startup concepts

      Bill: The other thing that’s really important to note, where the PR FAQ process is misunderstood, or, in conflict with the startup community today, is the lean and agile approach.

      Sonal: I wanna hear about this. Love it. Cause some fights, Bill.

      Bill: Here, let me tell you what the problem is with the lean and agile approach.

      Sonal: I love Eric Ries, for the record. He himself is the first — and he’s actually said it on this podcast, that sometimes people get a little cult-y and follow the letters of the rule instead of the principles of it. I actually personally am a big believer, having worked at Xerox PARC, in the maximum viable product sometimes instead of the minimum viable product.

      Bill: Yeah, it’s really — actually it’s the v part. The problem is that people focus on the m, the minimum, and they don’t focus on the viable. So, what is the definition of viable?

      Sonal: Yeah, what is the definition? What would you say it is?

      Bill: To me, the definition is like, “Oh, if I go build this thing, I have created a — insert size-of-business here — $100 million, billion dollar business — I have enabled, if I’m right, then this opens the path for, like, something really big. And I see instead happening, “Okay, I’ve got a sprint, I’ve got a couple of weeks. What can I get done in a couple of weeks?” Or, “Oh, what can I launch quickly to sort of test and learn?” Right, where then, you’ve created completely different constraints. And oh, by the way, if your whole dev team thinks with their whole roadmap, and breaks it into these little chunks of how do I you know iterate quickly, test, and learn, then you’re gonna launch a lot of small things where the actual size of the viable business on the other side of it might be sized in the one-million dollar range or two-million dollar range, or, like, the actual potential good outcome is very, very small.

      Now, there are plenty of places where this approach is totally applicable. Search, where you’re, like, how do I test and learn with, like, changes to the algorithm? Or changes to the logic, or a new AI model. But if you’re thinking about, I’m starting from scratch, I’m trying to create a new business — the problem with this MVP approach, where the viable part isn’t really thought out, and mapped out, is that people haven’t really thought through, like, well what could this really become, and why might this not work? And in many cases, they could have actually — if they instead spent more time upfront in the planning process — thought much bigger about what does the customer really need — or wow, I’ve really created some significant value versus frankly, these little incremental changes that don’t really even move the needle at all.

      Sonal: People don’t do press releases for incremental releases, they do press releases for big advancements.

      Bill: That’s actually an excellent point. If you parachute into a new company, and you look at their product roadmap, and there’s not one thing on there you’d write a press release about — then, like, you’ve got a problem.

      Sonal: Yeah.

      Colin: To me, viable means you read the press release of what you’re trying to build, and you want to buy or use the product. If it’s not something that you wanna buy or use, it’s not viable.

      Sonal: Yeah. I mean, I would also push back on the definition of viable, because what I heard from Bill — and Bill, you should correct me if I’m wrong on this — Colin, when you say like that obviously the customer is gonna wanna use it, but, to me viable is not just that it’s something a customer would use <Bill: Right!> because I think there’s a lot of dumb things customers would use, quite frankly. I heard it more as the enabling conditions to really make something bigger, and then also to address a deeper, more underlying opportunity sometimes instead of the surface opportunity. Because — not to get all jingo-ish on this — but I, of course, think of Clayton Christensen’s Jobs-to-Be-Done framework — and it’s sort of about, like, what is the underlying job to be done that is being fulfilled for this person? So, that’s how I heard the viable — what does it take to make this happen — whether minimum or maximum — and then also not only thinking about the product, but all the enabling conditions to get to addressing the deeper opportunity.

      Bill: Yeah.

      Colin: Another way where I’ve seen that MVP process being misused — because it can be useful — is that the process itself becomes the goal. And because I’m supposed to launch every two weeks, or I have an MVP so this MVP card is “go to the front of the line and get whatever resources I want because I’m doing an MVP.” So, don’t let the process itself become the end goal, and then don’t let the MVP get in the way of understanding your customers. Because you know, some people actually forget the customer problems that they’re trying to solve.

      Sonal: Which is the whole point of doing an MVP in the first place. I mean, the whole point is to be able to get that feedback so quickly. Because the whole reason that that idea came about was that, you don’t wanna have that very long delay in the feedback loop.

      Colin: Yes. And if you’ve watered down the feature set so much just to conform with what the process is, it’s a shallow, pathetic version of what you’re actually trying to test. And so you say, nothing here — when you never really gave it your best shot.

      Sonal: What’s really interesting is you actually, in your book, outline the total addressable market, or TAM. And I found that to be very interesting — especially in connection to the discussion about how to think about the biggest possible market for this product, working backwards.

      Bill: Yeah, we spilled a fair amount of ink on the TAM part in the book, because people with less experience sort of get this part wrong — to not only think about how many people have this problem, but like how big is the need. And for how many consumers is this problem really big enough that they’re willing to spend money to do something about it? How much would they be willing to spend? And, you know, how many of these consumers actually have the characteristics, or capabilities to actually make use of this product — and, a lot of people don’t really step through that clearly.

      Colin: The thing is that you wanna ask all of the hard questions upfront. And as we talked about, this is an iterative process. So, in the review meetings, you know if there are a couple of questions that you don’t know the answer to — you just append those to the document and say great, let’s come back in two weeks — however long it’s gonna take. And the FAQ, you can basically break it up into two different components. One is an external FAQ — that you’re explaining how does this product work, how much does it cost, why should I use this service versus what’s out there on the market, or why do I need to change my behavior. And then the internal FAQs — what are the things that we need to go organize and solve in order to make this product or feature a reality.

      And then, like you touched on — how big is this opportunity? You know, what is the total addressable market? Then you can size up the opportunity, and that’ll tell you whether it’s worth doing or how much to invest in it. Because one of the things that you ask when you go through the working-backwards process is, [is] this big enough to be worth doing? So it may be a good idea, but it just may not have the impact and scope on your customers or the organization to make a difference to be worth devoting resources to.

      Bill: When we were figuring out in digital media and AWS what are we gonna go build, and you parachuted into either of those teams and said, like, “Hey guys, how’s it going?” and we said, “Oh my god, we’re so frustrated, because we just wanna go and build this thing and get it out there, but Jeff, you know, is insisting that we go through this PR FAQ process and figure this all out in advance.”

      And I was among the people who found this frustrating. And it was only in hindsight that I realized, like, how smart Jeff was to slow us down. And we co-opted the marine scout sniper saying, which is that “Slow is smooth, and smooth is fast.” Or, “measure twice, cut once” — or sometimes Jeff would refer to himself as “the chief slowdown officer” when teams were sort of itchy to pull the trigger and build and launch something.

      What I came to appreciate in retrospect, is that to build — to actually create value, you really have to take the time and think big — that’s what the PR FAQ process does — and if everyone can go write PR FAQs, and you’ve got 30 or 40 different ones to review, then I’m here to tell you that the best ideas are gonna bubble to the top pretty clearly, based on this process. Whereas the MVP/lean process really is just about cranking stuff out, rather than first filtering, thinking through, refining…

      Sonal: What to do.

      Bill: …what to do. Yeah, people confuse speed and activity, with effectiveness.

      Principles in balance

      Sonal: Yeah, exactly. What I find so fascinating about that, by the way, just coming back to where we started, is how the 14 leadership principles balance each other. Because you describe Jeff in this context, and a lot of these processes as ways to slow things down up front — but then you also have principle #10 (or whatever number it was), for bias for action at the same time. You think of it as a whole system. And it’s really interesting, because it resonates to me with Ben’s — Ben Horowitz’s book on culture — because he describes, like, the value system of the samurai warriors, where they would have something that was like Bushido — where, there’d be something that was incredibly kind and generous, and then something that was incredibly vengeful in the same, like, framework. It’s very interesting how those motions are kind of opposite, but yet in balance.

      Colin: Yeah, they do work together. We started off the conversation about the “are right a lot” principle — well, the one right below that is “learn and be curious.” And the one right above it on the list is “invent and simplify” — and, you need to do both of those in order to be right a lot.

      The one thing I would add — you know, a lot of people say, well, Amazon, they either have so much money or so much time they can afford to do all these things. Long-term thinking doesn’t necessarily mean it takes longer to get to your end goal. You know, Amazon built a $100 billion business faster than any company. AWS got to 10 billion dollars in revenue from zero, faster than Amazon the retail business did.

      So, sometimes, to slow down to move fast — even with smaller organizations, [you] more than likely will get to where you want to be quicker if you take some of these steps. Typically, what you’re doing is you’re doing off-path, distracting activities, that by the way are taking away from your bottleneck resources — which are typically software engineering resources at a lot of organizations <Sonal: Yes, exactly>that are meant to build up long-term value.

      Bill: Yeah, and just to — the AWS thing is so remarkable. Think about that for a minute, because we just told you — I mean, what was it, Colin, 18 months, 24 months? How many months from go, before the team even wrote the first line of code?

      Colin: I mean, it was at least 18 months. And I did have software engineers after some of the meetings come to me and say, “Hey, can you remind Jeff that our job is to write software code, not documents?” and they were, you know, itching to go. It took a while to figure out what we should build, and that time was very well spent writing Word documents versus writing C code.

      Bill: And not only that, it’s empirical fact that it was well spent, because the company set the record for the fastest company to a revenue milestone by spending the first 18-24 months planning.

      Colin: And you talk about distractions, we did not know at that time what other people were doing. You know, web services were no secret. There were other companies who — by their own right — should have gotten there first. We didn’t know if someone would come out with a set of developer APIs at that point in time.

      Sonal: I know. That scares me thinking about that, actually. Like, almost two years planning, like, that scares me.

      Bill: Think how scared they were.

      Colin: But Jeff had the fortitude to say it’s not ready yet. This is not what we want to build.

      Bill: It’s not viable.

      Colin: You know, talk about what makes Amazon special, you can read these principles — sticking to them is sometimes quite challenging, either when things are going really, really well, or when they’re not going well, or when there’s uncertainty. These are not just posters on the wall. They’re woven into the DNA of everyone who’s been in Amazon for any length of period of time.

      Sonal: I’m just struck at all the parallels in all the things you guys are saying — because, essentially, every business today is a creative business. What really strikes me is — I’m not gonna make a comparison between podcasts and AWS, but I will say that I talk to a lot of podcasters [about] how to make their podcast stand out, differentiate. Like, how to do editorial strategy comes up a lot — and one of the things I constantly say is, if you can’t be first or leading, then you have to be very differentiated. It’s really interesting when you talk about the bottleneck resources, I think of things in terms of opportunity cost and return on energy, or what I call ROE. And, similarly, you have to pick which things you’re gonna do for the greatest possible hits, or you’ll never punch above your weight, or be heard above the noise. Which I think is so fascinating;

      We had Jeff Lawson from Twilio on the podcast recently, and he’s, like, you know, we need to give them problems, not just like specs and things to work on — and so when you describe that they’re like, “Uh, we’re spending, you know, 18 months writing planning documents,” that is treating software developers as creatives. 

      Colin: It also creates alignment for the teams then who will go out and build — because sometimes it requires more than one team. If they’re not aligned on what the problem is, they’re gonna solve different problems and those components aren’t gonna fit.

      Advice for startups

      Sonal: Right. I want to switch into talking about some specificities, some advice for startups. 

      Colin: One tool you can use for startups — and I’ve actually done this at other companies — if you’re trying to figure out what to build, you can write competing press releases, with different takes on the problem. <Sonal: Oh, I love that> <Bill: Right> And then, it becomes pretty apparent when you’ve got two or three or four different approaches that, we really need solution A. Or, I’m gonna take the first part of this solution and combine it with a great idea that came up in the second one — and this is actually what we want to build. So, it’s a lightweight way to crystallize your thinking and also get alignment.

      Bill: Yes, think about how inexpensive it is to write a one-page press release versus how expensive it is to build mockups. To Colin’s point, when you read them all, as a group, like, the best ones are gonna be clear.

      Sonal: You know again, this is where I just can’t get over how creative the book is, in terms of its application to all kinds of creative fields, companies big and small — because to me, both the memo strategy, the PR FAQ — and then this idea that you both shared of being able to write competing press release and then harness kind of the wisdom of the group — it’s actually about the power of narrative, for really helping instantiate things that you know when you know. I think a lot about how tools change our thinking and vice versa, and you guys are essentially describing these practices for how that plays out in organizations, especially as they scale.

      On this note, you have a section in your book — the subhead is “Better Coordination Was the Wrong Answer” — because that’s one of the common myths that people have as they scale is, we need to coordinate better. And in fact, it creates layers and layers of crap to deal with. People create entirely new roles, like dedicated chiefs of staff just to, like, you know, create stitching and seaming between teams. And it’s just the most ridiculous thing I’ve ever seen and heard. And I just wanna hear more from you guys, especially because a lot of listeners — and we’re talking about what happens when you scale and grow very quickly — you guys really did see this phase at Amazon.

      Colin: Yeah, this is a case where Amazon faced the same question that a lot of growing organizations do — we’re adding more people, it just seems like it’s taking longer to get things done — but came up with a different answer. Some companies would say, let’s build coordination tools, let’s collaborate better. And Jeff said, I’d love to eliminate coordination altogether.

      Now, in practice, you can’t eliminate it completely — so, you break up into loosely coupled teams. And this was a hard, hard problem to solve, because it required to change the way we built Amazon, technology-wise, to decouple it into what’s now services-based architecture. That was hard to do back then in, you know, 2000-2001, especially as you’re growing super fast. But then there’s also the organizational component too, about how do decisions get made? It was a multi-year effort, to be quite honest, but if you’re gonna go grow 10X and 100X, we’re not gonna spend any time building — we’re gonna spend all of our time coordinating. So, you know, nip it in the bud. And Jeff wanted Amazon to be a place where builders can build.

      Bill: So, the ironic thing is that we started off with the process that was the conventionally accepted traditional wisdom — it was brought in what we called NPI, new project initiatives — it was — it was horrible. I mean, basically it was another example of the process becoming a thing, where you had to come up with, like, what’s your new project initiative? You had to write it all up, you had to project what are the financials behind it — most of which were totally wrong, and guesses, like almost all pro-forma P&Ls are. And then put it in front of a big committee, and try to take it upstairs.

      Not only was this process a massive waste of time, but then you’d have these frustrating business reviews with the team saying, “Yeah we really need to go build X, Y, and Z,” and Jeff and the S team saying, “Yeah, I agree, you know go build X, Y, and Z,” — but they’d say, “Well I can’t, because I need browse to do this. And I need the team that works on the checkout — order pipeline, and they already have these four other projects they’re working on, so I’m just sitting here in line, and I can’t go build it.” And so, you’ve de-empowered your teams. The senior management, they become like referees of, between teams, who’s gonna go do what. And, frankly, it’s not much fun. So, this was a huge breakthrough for the company, was breaking down — you know not only, of course, breaking down the code into APIs — but then breaking down the teams into single-threaded focus teams.

      Colin: There are some roles that don’t actually fit too well in this type of paradigm. So, for instance, like a Chief Product Officer doesn’t really fit in this role, because if you ostensibly are responsible for making every product decision in the company — from what’s going on in the warehouse to what should the — how should the apps be built, to how can we decrease delivery time — there’s no one person who can be obsessed with all of those details. And so, that role typically doesn’t fit as you separate into these small, separable, single-threaded teams. The fallacy is [that] as your company grows, that Chief Products Officer isn’t really doing that anyway, because they can’t get that much high-quality information to make all of those important decisions. You do want the teams closest to the customers making those types of decisions anyway.

      Sonal: Yep. I call that “bare metal decision making”, like, who’s the closest to the metal of the thing. And that’s what I think is super valuable about what you just said, and you’re essentially describing these small units as, like, every team has its own mini – like GM, of every mini unit as a leader, and then every product — you’re, like, so close to the core in that decision-making framework.

      Bill: That’s right, the idea is for each one to be like a self-contained unit. And in search, that’s all engineers — but in some other business like my digital video business, it was like a combination of engineering and marketing people and people working with studios. But you have all of the resources you need — you don’t have these dependencies that basically slow you down.

      Colin: During this timeframe where we’re making the transition, we were using narratives, and one of the questions that we actually required was for the teams to put in “what things that are not under your control that you wish you had under your control” — and how <Sonal: Oh, great!> are you gonna organize and create APIs so that can happen? Again, this is work that’s below the tip of the iceberg of where Amazon really put a lot of effort into. How can you still grow and be as nimble and as agile as we were when we started out. And, it’s not easy — but, you know, it only gets harder, so you may as well start now.

      Bill: Yeah, and don’t take away from this that Amazon was some Nirvana world, where you never had to worry about coordination with other teams. It was just that we worked so hard to minimize the degree to which we did.

      Sonal: Yeah. I’m very fascinated by how organizations evolve in general, to be effective, and when you think of any large company, it becomes a complex system. And you’re essentially describing how modular, self-contained units can thrive in these complex systems. It’s a lot like evolution, really.

      Bill: And these pieces all fit together, because then we have to move back to like, oh, the reason that Amazon could do that is because those teams wrote narratives and PR FAQs, so they made it abundantly clear. If Jeff wanted to see what is this team doing, “Oh, here you go. Here, read this document” — and in, you know, two hours, he knew exactly what they were doing.

      Sonal: Right, he makes him the chief ecosystem officer, essentially, not just the chief slowdown officer.

      Shadowing Jeff Bezos

      Okay. So, Colin — so a question I have for you. It’s really unique that you were able to shadow Jeff Bezos, and be his shadow. And you know, I don’t wanna make this about the glorification of Jeff Bezos. I’m so not interested in that, because I think there’s just plenty of narratives out there — what I am interested in, however, is this question of what it takes for a CEO and a leader like that to evolve. And what you saw in shadowing him on that front, and then also the act of shadowing. And that itself as a mechanism for learning, mentorship, apprenticeship, if you will. I’m very fascinated by this whole thing.

      Colin: Sure. So, I mean, my role as Jeff’s shadow was primarily two parts. One was just to make him a more effective CEO on a daily basis. Making sure that the right issues are surfaced, that the people coming into the meetings would cover the right topics and have enough information to make the right decisions. And afterwards that it gets followed up on. So, you know, kind of bookending the day. But then the second and more important part, the longer-term part, was it was a training role. And the way Jeff put it is, I want us to be able to model each other, and you know how we would think in different situations.

      One of the enlightening things that I realized during my time with Jeff is what he chose to work on during the two years I was his shadow or technical advisor. And it wasn’t the biggest businesses at Amazon — about half of the time were spent on what would become AWS and Digital, and those businesses had revenue of effectively zero for, you know, those two years. And he told me more than once, you want your top leaders and your most impactful people working on your biggest opportunities. And that may be different from what your biggest current set of activities are.

      How Jeff changed during that time and just to become a better CEO — one thing he said is, he learned how to become a better operator, becoming more operationally efficient. Fortunately, that is a teachable and learnable skill. And Jeff had some great people at Amazon, like Jeff Wilke, you know, who’s a great operator. He’s the CEO of the Consumer Business. And now in his own right, he’s this great operator and insists on the highest standards — it’s one of Amazon’s leadership principles. He holds people accountable, himself included, by the way.

      And I would say that I also learned that sticking to these core principles is harder than it sounds, but the time when you need them most is the times when it’s easiest to ignore them. So, for instance, Amazon Prime, he made the call that, “Hey, we’re gonna go launch this shipping project in the holiday season.” And it was not very popular at the time, but when he explained his thinking, that you know customers were basically giving us a B minus on our 3-5 day shipping — even though we had just gotten reasonably competent and spent a couple of a hundred million dollars doing that — Jeff looked at it, you know, long term and said, “Well, we’re becoming a smaller and smaller share of the overall ecommerce industry, so we’ve got to make the change now.”

      So, it wasn’t that Jeff had this insight that comes once in a generation for Amazon Prime. He just stuck with the leadership principles and took them to their logical conclusion. When it was tough to do in the face of the holiday season, of what our quarterly results were.

      Sonal: By the way for the listeners, because we don’t obviously have time to go into the whole book, but, you know, half of the book — the first half is about the principles and these practices and mechanisms — but the other half is actually showing, through these very detailed case studies and examples that you both participated in or witnessed or oversaw firsthand, in this invention machine at work. And that includes the Kindle story, the AWS story, Prime Video, and what you just mentioned, Colin, which is Prime. And what’s really interesting in that Prime section — is that not only that Jeff had the wherewithal to push through — but what’s really interesting in the Prime program story is that it’s all the iterations involved to actually get it to what it is today. Because there was, like, a 1.0 and then, you know, the loyalty program it later became. So, I think that’s a really great part of the book, for those that want to learn more, and — Bill, anything to add there on what you saw on the evolution?

      Bill: I do, in fact. Jeff — I remember Jeff, at least more than once, talking about the way a leader needs to evolve as the company grows. And I think about this frequently, which is — he said, at the beginning, the leader needs to really focus on what. Then they really need to focus on how. And then, eventually their focus really just becomes who. <Sonal: Wow.> What is, you know, what is our business? What’s our product, what are we building, what are the details of that? How is, what processes? How do we do the work? What is the filter, the lens through which we make decisions? And then who, of course, is who are my leaders? Who, how have I — making sure that I assembled the right team — and in doing so, how do I delegate responsibility to those people so that they can carry out those details?

      That is a classic and challenging transition for any entrepreneur/owner, who starts off being in control of every detail, and that they have to slowly let go of those details. And then they have to figure out how they put in place the right mechanisms to be able to properly audit the work of the teams. And that’s what, you know, we tried to describe in the book, is actually this management science, frankly, that Jeff and the team developed to really solve this problem.

      Sonal: “Working Backwards: Inside Stories and Secrets from Inside Amazon” by Colin Bryar and Bill Carr. Thank you so much, you guys, for joining the a16z Podcast.

      Colin: Thank you.

      Bill: Thanks, Sonal.

      • Colin Bryar

      • Bill Carr

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Anatomy of a Hack: SolarWinds and Ripples Beyond

      Steven Adair, Joel de la Garza, and Sonal Chokshi

      In this special “3x”-long episode of our (otherwise shortform) news analysis show 16 Minutes — past such 2-3X explainer episodes have covered section 230, Tiktok, GPT-3, the opioid crisis, more — we cover the SolarWinds hack, one of the largest (if not the largest!) publicly known hacks of all time… and the ripple effects are only now starting to be revealed. Just this week, the U.S. Cybersecurity and Infrastructure Security Agency shared (as reported in the Wall Street Journal) that approximately 30% of both private-sector and government victims linked to the hack had no direct connection to SolarWinds. So who was compromised, do they even know, can they even know?!

      Because this hack is a supply-chain compromise involving various third-party software and services all connected together in a “chain of chains”, the knock-on effects of it will be revealed (or not!) for years to come. So what do companies — whether large enterprise, mid-sized startup, or small business — do? What actually happened, and when does the timeline really begin? While first publicly revealed in December 2020 — we first covered the news in episode #49 here when it first broke, and there have been countless headlines since (about early known government agency victims, company investigations, other tool investigations, debates over who and how and so on) — the hack actually began not just a few months but years earlier, involving early tests, legit domains, and a very long game.

      We help cut through the headline fatigue of it all, tease apart what’s hype/ what’s real, and do an “anatomy of a hack” step-by-step teardown — the who, what, where, when, how; from the chess moves to technical details — in an in-depth yet accessible way with Sonal Chokshi in conversation with a16z expert and former CSO Joel de la Garza and outside expert Steven Adair, founder and president of Volexity. The information security firm (which specializes in incident response, digital forensics/ memory analysis, network monitoring, and more) not only posted guidance for responding to such attacks, but also an analysis based on working three separate incidents involving the SolarWinds hackers. But how did they know it was the same group? And why was it not quite the perfect crime?

      image: Heliophysics Systems Observatory spacecraft characterize, in the highest cadence, the constant stream of particles exploding from the sun affect Earth, the planets, and beyond via NASA Goddard Space Flight Center / Flickr

      Show Notes

      • An overview of how SolarWinds was hacked [2:21], the attackers’ methods [5:33], and their impressive sophistication [8:17]
      • A step-by-step explanation of how the attack took place [14:20] and how the hackers avoided detection [21:18]
      • Open discussion of what the experts know so far [23:52], including how we know the attack was coordinated by the same group [29:21]
      • Big picture security questions [33:26] and how businesses and consumers can protect themselves [42:51]

      Transcript

      Sonal: Hi, everyone. Welcome to this week’s episode of 16 Minutes, our short form show where we talk about the news, tech trends in the headlines, tease apart what’s hype/what’s real, and where we are on the long arc of innovation. I’m Sonal, and today’s episode is actually one of our special “2-3X” long explainer episodes — which I’ve done every so often for topics that keep coming up over and over in the news (most recently on Section 230 and content moderation, previously on TikTok, and even earlier on on the opioid crisis). You can catch all those at a16z.com/16Minutes. But today, we’re covering the SolarWinds hack, one of the largest (at least publicly known) hacks of all time.

      Not only has it been in the news a lot since it was first publicly reported in December, with countless headlines since — but the most recent report, from the acting director of the Cybersecurity and Infrastructure Security Agency, was that approximately 30% of both private-sector and government victims linked to the hack had no direct connection to SolarWinds, as reported in the WSJ just yesterday. So, it’s gonna have ripple effects for quite some time.

      So, we’re doing an “anatomy of a hack”: a teardown of the specifics we know so far, what went down, and what we need to know — whether big company, small company, or individual.

      For quick context before I introduce our experts: Over 18,000 customers downloaded compromised software, though it goes well beyond them. Those customers include several large government agencies (which we covered last year on this show). Private sector victims include companies like Cisco, Intel, Microsoft, NVIDIA, Deloitte, VMware, Belkin, and others. The broad consensus – per a statement issued by the Office of the Director of National Intelligence, the FBI, Department of Homeland Security, and National Security Agency – is that Russia was most likely the origin of the hacking, and more specifically, that the Cozy Bear group (also known as APT29, overseen by Russia’s intelligence service) was responsible.

      That’s just a super high level, because we’re actually gonna go deeper to break down the who what when how – and the chess game of it all. So now, let me quickly introduce our experts. Our in-house expert is a16z operating partner for security and former CSO, Joel de la Garza. And our special expert guest is Steven Adair, the president of Volexity, an information security firm that does incident response and forensics (including memory forensics), and they’ve responded to multiple cases of this. Their team actually put out several detailed posts on it and more.

      Overview of the SolarWinds attack

      Sonal: But first, Steven, can you summarize what happened? Obviously we’ll continue to dig in on the details throughout the episode, but the reason I’m asking is, I’ve started to lose track. I bet a lot of our listeners are getting a little inundated with this headline fatigue too, like — now this, now-what. So, tell me basically what actually happened. What do we know?

      Steven: Yeah, sure. So, SolarWinds is a company that creates network and system management software that’s used really heavily by tens of thousands of organizations around the world, so it’s used by large giant commercial companies, Fortune 500. It’s used by small organizations, managed-service providers, and governments. So, it’s a piece of software used to manage these, like, really sensitive important assets. So, think about the IT teams, and people who wanna watch what’s going on key systems, on network devices, and things that are really important within a network. They have a product called Orion, that’s their flagship product.

      And what happened is that SolarWinds was basically breached. How exactly, you know that’s not really been published. We don’t know. But attackers were able to compromise SolarWinds, get into what’s called, like, the “build process” of this product. So essentially, the development or the software that’s downloaded and used by all these organizations, they were able to get into SolarWinds’ networks, and modify that build process.

      And what’s interesting and notable about this, is, they didn’t go in and modify the source code. What they did is — think about if you’re on an assembly line, and someone made a change like early on, and they all put it together — they actually waited til the very end, the very last step of compiling this package to make this software. It goes out. And they monitored it, they watched it, they looked at it, they learned, they tested. And they ended up compiling in a backdoor — which would give them access to the systems running SolarWinds Orion — for anyone who installed the update, or downloaded it freshly since they did this.

      So, they were able to modify SolarWinds and push out this update to organizations all around the world. And basically, they’d create a shopping list and selectively target who it was that they wanted to go into, and basically break into and further their access. They could look through and see, “Oh, this company, or this government agency, I’m very interested in them.” They could actually activate and walk right into their network, and they’re already sitting in and going into a very sensitive part of that network.

      So in short, it’s what’s called a “supply-chain compromise,” where, they’re really in the build process. Insert themselves to the backdoor into this legitimate software and expand their access — and do it very stealthily for many many months — until, you know, FireEye came forward and figured this all out in December 2020.

      Sonal: Right. And just to quickly, even more high-level-context it — this is playing out against the broader landscape of — for many years now, companies have obviously been using various providers of third party and cloud software and services. We’ll delve into this whole notion of a supply-chain hack, what it means, what it means for the future of security.

      But the thing I wanna really pull on from what you said is that this was very unusual because they didn’t go for the source code, they kind of waited for the updates, and then they were very targeted — as opposed to just sort of spray and pray. So, in your assessment of all the hacks that you’ve seen out there — and Joel I wanna hear your thoughts too here — is this really a sophisticated hack? Because obviously, in our show, we not only tease apart what’s hype/what’s real. I often wonder if that word gets thrown about very casually.

      Steven: Yeah, so from our opinion, it’s definitely — this aspect of it — is certainly one of the more sophisticated that we’ve seen. And it’s not necessarily that there aren’t a lot of smart people around the world, good and bad, that couldn’t pull off something similar. It’s, you know (1), the fact that they did; (2), they did it so strategically; and (3), you know, even if they had gone in and modified the source code, people would still be talking about how sophisticated it was. But, they took it up a notch and basically said, “Yeah, we modify this code, or someone’s watching it, or they audit it, or someone’s watching a check-in process.” Basically it went to a system where none of that mattered anymore. And they just kind of bypassed all that and went, like, straight for the jugular, in what would — I would argue a much more difficult way to go about it, but a lot more likely to meet with success and go undetected in that. I think they gambled correctly in this case.

      Joel: I mean, I think with these kinds of operations — and this is ultimately an espionage, you know, nation-state professional-type operation — from my perspective, the duration and the extent to which these things can run undetected is usually the indicator of how sophisticated they are. And so, like, these long running, you know, really successful campaigns that avoid detection really belies like a level of sophistication.

      Because operational security, right — like, covering your tracks — is actually just about as hard as getting in. And so, you know, the fact that they exercised their ability to cover their tracks for so long, to know where to insert in the process, and to lay low, is just indicative of a level of discipline that you don’t necessarily see in a lot of attackers.

      Sonal: Not just get in, but be able to cover their tracks, which is what both of you guys say. And by the way, we’ve only talked about the duration of when the hack was revealed by FireEye, and that it had been you know several months before. Do you guys have specifics on what the latest date-point is, in that timeline?

      Steven: Yeah, at first, essentially what they did was an experiment early on — and this has been posted publicly. The code in SolarWinds Orion was modified in late 2019. Where basically they made some initial modifications, which actually didn’t do anything malicious or put a backdoor, allow any type of access.

      Software went out. And they basically were able to prove, like, “Hey, I succeeded at doing this, it existed, no one noticed anything” — and essentially waited at some point to move on to phase two, which was, “Okay I can get in and go undetected, I can have it build, it all works, stuff makes it into production, no one notices.” And they said, “Okay, well, I’m satisfied with that. Now it’s time to, you know, go for broke and put the actual code in there, and open the floodgates.”

      Sonal: What you just described, Steven, sounds exactly the way a company builds a product. Like, “Hey, we’re gonna test it out. We’re gonna try an experiment, an MVP, a minimum viable product, if you will. Then we’ll, based on that, decide how to deploy it and target it, and blah blah blah.” I mean, I hate to say that, but that’s exactly what you just described sounded like.

      Steven: Yeah, it honestly wouldn’t surprise me if they had done some way of trying to basically clone their development environment too, and probably tested this — I would guess — probably pretty thoroughly before they even <Sonal: wow> ran the tests within their network

      The hackers’ sophisticated methods

      Sonal: So, they were incredibly savvy in certain ways, in terms of how targeted they were, and the choices they made.

      In the Microsoft blog post, one line in particular really struck me. It said that the threat actors were savvy enough to avoid giveaway terminology like backdoor, keylogger, etc. Instead, they gave their tampered code an innocuous name, “Orion Improvement Business Layer,” that would fit right into a marketing brochure. (This is from an Axios post summarizing it.) “The attack’s crucial door-opening exploit was a small chunk of ‘poisoned code’” — which is what Microsoft dubbed it – “all of five lines long or roughly 160 characters.” And then Ina Fried at Axios goes on to comment (which I had to chuckle, even though it’s sad), was, “This could well be the most damage per character yet achieved in the short history of cyber warfare.”

      So, I am curious if you have any thoughts on some of those — honestly quite clever — things that they did, to hide undetected. And any more specifics you could share there. And then we’ll go into the step-by-step in a moment, too.

      Joel: The fact that they’re not naming variables and naming things that are commonly used in attacks is mostly a credit to the existing kind of antivirus and anti-malware industry. You’ve got a lot of tools that are out there that are looking for this stuff. And you would imagine any adversary that’s relatively sophisticated is gonna run their changes through all those tools to make sure they don’t get detected before they deploy it.

      And so, that’s just table stakes for this kind of activity. It doesn’t really show any kind of real sophistication.

      Sonal: Of course, it just depresses me to hear that — and we’ll talk about this at the end, which is what companies and people can do. Because I’m, like, great — the better and better we get, the more and more sophisticated they get, and it just becomes this like never-ending back-and-forth, back-and-forth escalation.

      Joel: Espionage 101.

      Steven: Yeah. To be completely honest, that stuff doesn’t surprise, especially when their job is to, like, blend in as much as possible.

      But I’ll add to one of the things — and make sure that we give credit — some of the analysis of things we’re talking about today are obviously from — a lot of security communities have come together and published a lot of detail, which has been great. But this is one of the other things that they did, is, they actually used an existing config file that is part of SolarWinds Orion, that’s there legitimately — it was there five years ago, it was there two years ago, it’s there right now — but they actually repurposed that exact config file. They created a specific value and said, if this is a three, you shouldn’t beacon it, you’re basically turned off. And they use values in fields within this to then leverage that file that’s already being read and used by the program, to then also inform it on some of what it should do.

      So they use, like, native, existing files and functionality and things that are very innocuous-looking. And then they did a couple of other stuff beyond that, that are pretty stealthy – although they’re not necessarily rocket science, they are very uncommon.

      ~ One of them is the fact that this backdoor, once it’s loaded, it wouldn’t start its beaconing or calling out for this DNS activity (which I know we haven’t explained yet), but basically, the mechanism by which it actually gives that avenue of control back into the system. You have to meet certain criteria before [you can] even, you know, beacon. For example, if you weren’t “domain joined” — meaning you’re less likely to be an actual corporate asset. You’re someone testing it on a computer, you’re a workstation at home. You’re not even gonna pass the sniff test.

      ~ But what they then do is actually set a timer. And so, it might be actually up to two weeks before it actually starts doing anything. I might be under scrutiny from QA, or a build, or someone might be looking at it when they first install it, make sure it’s not malicious — so they actually say, “Hey, I’m just gonna wait two weeks. I’m in this environment, this is for the long haul, I’m not in a rush to immediately get access to these systems.” So that’s an interesting aspect. It’s actually fairly uncommon to see malware that is on any timer of significance, or driven by a specific event that’s likely to happen very soon.

      ~ The other thing that was really interesting: The malware basically would activate when a certain response was given to its query. “Hey, go connect to this domain name,” or “go connect to this website.” And, those domains that they used were actually domains that had expired. One of the telltale signs when you’re looking into malware and things is, like, “Oh, it was just registered last week or last month or earlier today.” So, this would pass that sniff test, all day long. Some of them had five or six years they had existed. It might even have, like, a website. They picked up infrastructure that had a history to it. They actually owned and controlled these domains. They weren’t, like, hacked domains or things like that, where they were using compromised infrastructure. So, just kind of an interesting note on that front.

      Sonal: It’s interesting and, honestly, a little creepy. I got goosebumps while you were talking, because it makes me think of every long game. The patience, and waiting, and stalking — that really skilled predators do. And I don’t mean to glorify it by any means, but I am just sharing that what you just shared in technical terms — it gave me goosebumps, quite literally. I don’t know how you think about it.

      Steven: When we first saw this in July of last year, we had I think three domains that we had seen used in that actual attack. And as we looked into them, we said wow. Like, we kind of noticed it’s just, like, yeah, these things have a real history. You know, what the hell is going on here? And then we found a way to find more of their infrastructure (even if we hadn’t seen it used in the attack), and they all had this in common. Like, we had a way which we could figure out and find some infrastructure from some mistakes that they had made. That’s why in our post we actually were able to provide a lot of indicators. Like, DHS included that in their list and everything.

      But, other than that, each one of the domains we looked into, we just instantly knew at that point — I mean, we already knew we were dealing with an advanced threat actor, but — we were kind of thinking to ourselves, like these guys have really stepped it up a notch. This was actually the third time we had dealt with them in an incident-response engagement. But this was, like, a little bit different than the other two rounds. There’s a number of things that just made it stand out, and that was definitely one of them.

      Sonal: This might be the first a16z Podcast Network show to be optioned for a movie. I’m just gonna say it right here, on air. Joel, anything to add to that before I switch into the detailed step-by-step?

      Joel: I mean, only if Matthew McConaughey plays me. <Sonal laughs> No, I’m just kidding.

      Sonal: I listen to him on the Calm app every other night or so.

      Joel: Yeah no, I mean I think that’s exactly it. Just the level of preparation, and just the long game that these guys are playing.

      You know, this malware stuff is pretty common on the financial crime-ware type side, right, people trying to steal money. But those actors typically register domain names within a day, it’s just all very phish-y and suspicious. But to see someone build these, like, really advanced, large, complicated infrastructures, years ahead of using it — it just belies a real level of sophistication, you don’t really see every day.

      How the attack took place

      Sonal: Okay. So, just to recap for listeners where we are and where we’re going — we’ve covered what happened at a high level, including some of what’s hype/what’s real, and interesting or undercovered in the media.

      You did a great job summarizing, Steven, but let’s now spiral into that a bit deeper and fill in some blanks that you haven’t covered. Both technical details — you mentioned the beacon, DNS — I want all of it. How folks figured things out — so we can then know what the open questions still are, ripple effects and implications, and then more on supply-chain compromises and what we can all do. But I especially want to know the anatomy of how they got access to the emails. But start from the very beginning of the timeline.

      Steven: Yeah, so the story of the SolarWind supply-chain compromise obviously starts with SolarWinds — and that’s probably where some of the question marks are currently, and they might remain that way. They were breached sometime at least as of late 2019, and then ultimately — what came out later in May of 2020 — pushed out an actual backdoored version of their software. A backdoor meaning, a piece of software that shouldn’t be there, that allows this foreign adversary to have control or remote access into these systems. So we’re talking in late May, that happened. From the cases we’ve been involved in and things that have been published publicly, we’re seeing that a lot of the threat activity started in June and July.

      The SolarWind software would send out this DNS query. So, when you want to go to a website, you wanna go to a16z.com, you type that in. There’s a system called DNS, it says, “Hey, where is this located?” A DNS server says, “Oh, it’s located over here.” It’s the basis [through] which kind of you can find things on the internet, so you’re not memorizing these numeric IP addresses. 

      So, the malware — all it did, once it finally activated — it waited between 10 and 14 days before it would start creating these DNS queries — it would do these DNS queries from the SolarWinds Orion server. And those DNS queries contained encoded data. And if you decoded that data, it gave you different information, but one was information about the network that that machine is joined to. So for — in the example of, you know, say, Microsoft, it might show Microsoft.com or Microsoft.internal. Or, you know, one of these government agencies, it might say treas.gov.

      But it would give this indicator, so that the attackers could actually see who these victims were — because remember, they were indiscriminately pushing out this software, potentially tens of thousands machines. That is an untenable thing to manage, and go and manually look at everything, and try and actually install software and do something of significance. And their goal is to stay under the radar, and not get caught. And then now they have to decide who it is they want to go after further.

      So, they probably have a shopping list that they started with, and they probably have a new shopping list of things — they’re walking into the grocery store and didn’t even know they wanted that, but now they know they do. And they essentially issued commands, and allowed them to initiate this backdoor on who it was that they wanted to attack. And they did this through a specific DNS response called a C-name value. So, it says, “Hey, where’s this host name?” It responds back. They would actually send a specific response to prep it, so that the malware would be waiting to know that next time something happens that it should take a specific action and open the backdoor.

      It would respond with these domains. And these domains would basically be the control points of where the attackers within have the hands-on keyboard — a human is doing this at this point. Someone says, “I am ready to take a look at this system,” and now hackers that are behind this are actually involved, and they’re saying, “Now I wanna look around and figure out. Is this a test machine? Is this a real network I’m interested in? Is this a lab environment? Is this a staging environment?” You know, things like that. And they can figure out, “Is this the real deal? Does this have access where I want? Do I want to proceed?”

      And they did this for — we don’t know how many organizations, and that’s the real scary part in all this, is — you have all these people that have come forward, and they’re, like, big companies or they’re these government agencies, and, that’s just the ones we know about. I don’t think anyone has a real notion of the size and scope of where they took a further interest and then actually did something. In our particular case, we got permission to write up and share details of our incident investigation. The attackers were very focused on getting access to email of specific individuals. So, their goal was maintain access, move around — you know, get what they need — having access to specific individuals, and what they’re writing, who’s sending them, why they’re communicating — was a key focus of what they’re doing. We were able to see that they did that.

      The interesting part, in kind of stepping away slightly from SolarWinds — and why the intel community and law enforcement says it’s likely tied to Russia (APT29, or the Dukes) — we’ve been tracking a group we call Dark Halo, just because we’ve dealt with APT29 on many occasions in the past, but we just have no real way to link the two.

      But what was interesting to us, is the story of this group didn’t start with SolarWinds. We worked three separate incidents involving these SolarWinds attackers, who we called Dark Halo — so, this is a story that starts well before, and has multiple other avenues.

      We had actually dealt with them back in 2019. We had an organization we were doing work with, and we kicked the group out. They went away. In our initial response, we had determined they’d been in that organization for 4-5 years prior. They came back in Q1 2020 through an Exchange control panel vulnerability. You know, mail service — they had a vulnerability that attackers will take advantage of. Got back in, stole email for certain individuals. They were kicked out and removed again. That’s what we did. And then they came back a third time with SolarWinds in July of 2020 again. We didn’t have a good way to prove it, and we took steps and mitigations in place to deal with it.

      So to say, “Hey, how did they get into, you know, SolarWinds,” or wherever else they’re operating — well, this isn’t their only trick. They have a lot of tricks up their sleeve, and they’ve been able to do this and operate for quite some time.

      Sonal: Wait, so how did you make that link across those separate incidents that it was the same group?

      Steven: I’ll tell you, and it was something interesting, is — if we had worked them at three different organizations, we actually wouldn’t have come to the conclusion that this was a single threat group. We wouldn’t have linked the three things.

      Any advanced attacker, anyone in network, they have certain commands and things that they’re gonna do — but they changed enough between each of the attacks, that the actual techniques, the tools — there’s a custom malware, or a commercial script, or a public script, like <inaudible> or a pin-testing framework, or these different toolings, or a web shell — they changed it between each one of the hacks, where it was able to be very non-obvious it’s the same group.

      But what they did is they went after the email of the same people each time — and why we are 100% certain it’s the same group, is — when they would steal email. They would only take a certain amount of email. They would specify, “I want all the emails since the last time I took it.”

      Sonal: Oh, so it’s like incrementally building on the total — oh my God, that’s so fascinating. <Steven: Exactly!> Keep going, yes.

      Steven: So in early 2020 they got back in, and they said, “Okay, well, I want all the email for these particular users since, you know, a specific date in 2019.” And then when they came back in through the SolarWinds vulnerability, they basically said, “Hey, I want every email for these people, and I only want it starting from this specific date range starting in early 2020.”

      So, we had each time they came back and asked for the email since the last time they did it. So in the one case, obviously, they had an intimate and previous knowledge. The other cases we worked, they didn’t have as much knowledge. They had to work their way and kind of figure out the way of the land. So, we’re dealing with the same group in all three incidents — that’s an interesting tidbit.

      Sonal: I was about to say, I still have goosebumps. That’s incredible. That was so good, Steven.

      How the hackers covered their tracks

      Joel: Pretty impressive analysis and work there.

      The things that really jump out to me is, this is something that is linked together over a 4-plus year campaign, trying to maintain persistent access to the communications of high-value individuals.

      I think the other thing that really jumps out to me is that they have a big data problem. They got access to tens of thousands of computers, and potentially thousands of organizations. It sounds like the kind of analysis that Steven has done is pretty unique. There aren’t a whole lot of people in the world that can do that sort of thing. And so, this is probably an incident that we’ll be continuing to understand for the coming months, if not maybe years.

      There’s probably gonna be a really long tail on that. These people are still out there, they’re still operating. What are they doing now? That’s particularly concerning.

      Sonal: It’s interesting because Martin Casado — you know, our general partner, who’s also a security expert — he mentioned to me that he thinks it’s super interesting how interactive the attackers are during the attack. Because it’s obviously a very sophisticated team of people gathering data and making chess moves in real time.

      And it’s so fascinating because when we report and talk about and communicate these types of attacks, we kind of make it seem like it’s malware that does all the work — but it’s really the people that are at the center of it. And then on the other side of it, you have this whole interesting dance, on your end — as sort of this forensics expert with your team, going in, and trying to figure it out, and the puzzles, and everything involved.

      Joel: Well, you know, I heard chess is popular now.

      Sonal: <laughs> Queen’s Gambit, right.

      Joel: This is exactly like playing a game of chess. The difference is that you don’t see the moves immediately — they get revealed over time, and then you’re left kind of piecing other things together.

      Sonal: That’s exactly the analogy.

      Steven: Yeah, I definitely agree — that their goal was to actually not have their moves — what they did never be understood. You know, we noticed the versions of their software that were downloaded. There was an update to SolarWinds Orion — I believe it was in August of 2020 — and that version wasn’t backdoored anymore. It didn’t have the malicious code. So we initially speculated, “Oh, did the bad guys remove it? Did SolarWinds find it? Did it inadvertently get removed?” We didn’t know how it was going down at the time.

      So, they removed the code. They got in, got all this access, and basically said I’m gonna try and remove this now and, like, fly under the radar. So, if they had their way, they would have pulled off like the perfect caper, done all this stuff — no one would have known how it happened. And then the Orion product, basically, would have nothing malicious in it. <Sonal: wow> So, just a, kind of like an interesting other thing that they did.

      Sonal: It is. It’s a very vivid contrast to the analogy of chess, especially given the popularity of Queen’s Gambit, when you see them recording their moves, and the spectators watching — it’s a real contrast to this idea that you’re literally making the move, peeling it back, making the move, peeling it back — it’s really stunning.

      Open questions for the experts

      Okay. So my next question before we talk about some things we can expect to see moving forward — what are some of the open questions still on the table? Like, we know SolarWinds was compromised, but the big open question there is obviously we don’t know how. Then the second big thing in the Microsoft post that I saw (and Steven Sinofsky pointed this out), which is, you know, they do this outline, but we still don’t know how the signed code was signed, so that whole idea of “sign the code” is a bit of a mystery still.

      I want to hear from you guys, what are your open questions — or what are the open questions the industry is still looking at, or that people should or shouldn’t look at?

      Steven: Sure, yeah. So, how was SolarWinds compromised? Obviously one of the open questions. You could spend as much time and resources — you could use infinite resources, and you may not ever be able to answer that question because that system is gone. It was wiped. All the logs are here, that was never logged, or it happened five years ago. So, I would say the scariest part of this — people are finding out about this in December, for something that was operationally live in May. They had a looong headway into breaking into different organizations, doing that shopping list. And there are going to be — and there have been — from this very group, and as a result of the SolarWinds compromise, more supply-chain breaches.

      Some people are breathing a sigh of relief, “Ahh! I didn’t run, you know, SolarWinds Orion software. I’m safe.” That’s not necessarily true. We’re not trying to sow fear, uncertainty, and doubt that everything is untrusted — which arguably, you need to go to a typewriter, send pigeons now — but it’s IT companies, it’s security companies, it’s managed service providers, it’s managed security service provider. There’s these different people that were running SolarWinds that then had this level of access to either directly get into networks, get into email, get into authentication systems to provide software or software updates or software downloads. They 100% certain had access to numerous networks and systems that would allow them to rinse-and-repeat SolarWinds, probably on numerous different scales, in numerous different ways. It doesn’t have to be through a build-time compile. It could be, they change a download, they change an update process. They took keys, or secrets, or remote access protocols, or passwords that got them into like other networks or other systems.

      So, the scary part is, is that the supply-chain compromise here is just causing a chain reaction that’s probably already impacting other organizations that have no idea. I think that’s one of the biggest questions, is — who else was victimized that we don’t know about, and what do they do?

      Sonal: So what you’re basically describing is, like, this complex, adaptive system — like, everyone sort of networked and connected trying to tease apart the scope and ripples of this is gonna take ages. And we might never, ever get to the bottom of all of that, because of that connectivity.

      It’s interesting because General Paul Nakasone, or Nakasone — I’m not quite sure how to pronounce it — he heads both the NSA, the National Security Agency, and the military’s U.S. Cyber Command. One of the things that they talked about is that developing a coherent, unified picture — what you just described, Steven, of the extent of the breaches — has been difficult. The challenge is that, “He’s expected to know how all the dots are connected, but he doesn’t know how many dots there are, or where they all are” — which is kind of a distillation of what you just described. What are the other open questions that are on the table?

      Joel: For me, the big open question — and with all of these really sophisticated breaches, the first is, how many stupid things led up to this? Like, how many ridiculously, easy-to-solve problems, like applying security patches, or using two-factor authentication — like, how many of those kinds of things we know we should always do are responsible for this — is always front of mind when we see this.

      Because I think when you double-click on these, a lot of the times it starts off in a fairly innocuous way, which is, like, someone guessed an account, or someone got access to some account. But as this event shows you, if you give a sophisticated actor a toehold in your organization, they’re just gonna run through it. So, that’s the first one.

      And then the second one is, we think of these breaches — because of just the way the media covers them, and the fact that they kind of show up sporadically — we think of them as, like, events in time that have a start and finish. But, in reality, these groups are still running, and we’re still facing them. You don’t know the implications of any of this stuff for a while. Like, you don’t know if they were getting into the Department of Energy to read, you know, Rick Perry’s old emails, or if they were getting in there to steal futuristic bomb designs. Maybe there’s gonna be some new weapon that pops up in 15 years and it’s, like, linked to this breach. And we’ve seen from these breaches — like, if you go all the way back to some of the first ones that have been publicly reported — you know we’ve often seen that the goal of these is either to spy on individuals and get some intelligence there, or to steal the designs for things that people want to go recreate.

      Sonal: Right. And don’t forget that oftentimes — I think we often forget to talk about, when we talk about intelligence — it’s often in the form of blackmail, right? Like, we’re not just talking about stealing IP and obvious secrets.

      Because a lot of people dismiss this as, “Oh, email. I just book events and share, like, photos with the family in my email.” I don’t think they realize that it’s such a vector to all these ways of really exposing who you are. It’s your identity, in many ways. So, that’s another way to think about that too.

      Joel: Absolutely.

      Sonal: Anything else on the open question side?

      Joel: So, a bunch of other secondary breaches are now being reported on. Some of the Microsoft stuff, you saw that there were people creating reseller accounts, or trying to get reseller access to people’s Office 365 enterprises. And then there were certificates that were compromised for things like Mimecast, and maybe perhaps other services that are out there.

      And so, like, this picture starts to emerge that there’s these — lots of fires just started burning. And it’s always really difficult to tell if it’s one fire massing together, or just a bunch of different people that are acting independently.

      Sonal: That’s actually something I wanted to really quickly touch on before we go into the rest of this. Because the thing that was confusing to me is, okay — so, I read the Microsoft post. You know, like, there’s some intrusions. That there was a partner for Microsoft, actually, that handles cloud access services. We don’t know how connected or not connected it is. Then you have a reseller gaining access to Microsoft customers’ Azure accounts. Then you have this reported Russian state-sponsored effort exploiting a VMware flaw that the NSA warned about last month, that takes advantage of a recently announced vulnerability in VMware Workspace One access. Access Connector, Identity Manager, etc. And, this is according to the NSA, that they’ve had at least one case — that they’ve successfully accessed protective systems by exploiting the flaw.

      And then you have, like, you know, one after another, and they issued a patch. I mean, I am reading all these at the same time and I’m like, is it all the same thing or not? I think that’s what you’re saying, Joel, about — we don’t know if it’s all one fire, or a bunch of fires. And do you guys have any thoughts on how to connect those dots, if at all?

      Steven: So, as a general statement, I would say what we know about this attacker that we call Dark Halo — the people behind the SolarWinds hacker — they’re extremely adept in methods that allow them to gain access to email or systems involved with email. So, things like trying to get access to an Office 365 or Azure AD environment through a partner organization. Or by stealing some, you know, SAML tokens or some kind of authentication mechanism. Or, trying to get access through some other — possibly through a vendor — to get access to that same data to email data, essentially by any means necessary. I would say all of those are very on par with what we’ve seen this attacker do and focus on, and what others have seen. A very good chance that they are related.

      But even if they weren’t, it just kind of underscores that there’s a lot of people trying to get access to this data. And now you need to focus a lot more on the cloud, on the technologies that are used to secure the cloud or that have access into it. And the things and places where people don’t always look — because it’s new to them, or they never looked at it, or they didn’t know to look at it — so, I think this event will actually end up advancing security in many ways, because it’s causing people to think about and do things that they weren’t realizing before. And as you can see, the bar’s been set higher to where they can’t walk right in the front door anymore, right? They’re not easily able to get right into these organizations by compromising, you know, the core network or the system administrator and the other ways which you could get there.

      So, in some ways, it’s a sign that security has improved a lot — but also that there’s a massive amount of work to do at the same time.

      Sonal: It makes me again think of the chess analogy, and when you have a player that comes to the table that has a set of moves — like, patterns that are well beyond what the human mind can even comprehend — and that makes me think a little bit of even, like, AlphaGo playing Go with a real chess player in Korea. And how you know the system made moves that they considered very alien, but that a human being would never have done, but that still follow the rules of the game — the constraints of the game, that is — and yet were completely novel. And if you just keep seeing more and more moves kind of grow and become more and more sophisticated on both sides — even as we may improve, like, there are gonna be alien moves at some point.

      Steven: Well, to be completely honest, they’re undoubtedly highly skilled and disciplined — which, if you think about it — okay, if we go back to the chess analogy. You know, are they a master, are they a grandmaster? In some ways you can say, okay, they’re a grandmaster — but most of their opponents are unranked. So, they have this kind of lower skill, and their strategy is easier. But then they’ve been able to go to these people who maybe their security defenses are much higher ranked, and they’re using that skill set, that knowledge, and that kind of cat-and-mouse, to still get into those organizations. But to have to do that, it shows that people have leveled up quite a bit — which is a good thing for these companies and the security industry.

      But at the end of the day, they still managed to either capture that king or get them to knock it down. I guess no one’s really thrown in the towel. No one has surrendered, that I’ve seen so far. But, I would say they’re winning a lot of matches, and they’re playing a lot of them simultaneously.

      Sonal: Right. But they are not (to be clear, to your point), an alien player, like an AlphaGo. They’re still moves that are human, just very skilled.

      Steven: At least from what we’ve seen, but who knows, like, what we’re missing though, right?

      Broader security trends

      Sonal: Right. Okay, so now the big picture questions. We’ve covered what happened, how it happened, the details. We talked about this, you know, phenomenon of supply chain attacks, chain-of-chains, what it means. I would love to hear what you think about this, when you think about the broader trends at play.

      Joel: Yeah, absolutely. I think on the podcast several times — I know I sound a bit like a broken record — but we’ve talked about the biggest challenge being securing the supply chain. And how all these businesses that are becoming software businesses are actually becoming reliant on other people’s software. And so, it’s not just a matter of the stuff that you write to run your company, it’s also the matter of the stuff that your suppliers are writing.

      And as everyone knows, security is really difficult, and it’s hard to secure your own things — and then having to worry about the security of your suppliers is adding an additional layer of complexity.

      And so, over the last couple of years, there’s been a lot of investment in trying to understand third-party risk management, vendor-risk management. How to glue these things together. There are several different approaches, everything from private systems that will look for vulnerabilities and report on the risk. There are publicly available standards, different trade groups are trying to develop their own standards for security — and then certain vendors are trying to come up with their own standards. There is no easy answer, and so what you’ve got is a lot of different approaches that are being tried, and a lot of experimentation that’s taking place. This is probably the first breach at such a size, scale, and scope. So this is kind of the watershed moment for that third-party risk management.

      And there’s any number of other suppliers that are out there that are in very similar positions, right, and it could be a company like SolarWinds, or it could be an open-source repository that a bunch of people are building into their applications. There are any number of different ways.

      The thing that’s really difficult for me — based on where I sit and what I see — is if you play through all the different potential solutions that are out there, it’s really hard to know which one of them would have actually prevented this? So, like, if I went to any of SolarWinds’ customers and said, “Hey, what’s your vendor risk-review report on SolarWinds?” You know, before the breach, I’m sure they would have said it was a wonderful company, it was doing everything, they passed our review, they answered our questionnaire. You know, they’ve got the people hired, they have a program. And so, it really comes down to how do you actually measure these things, and how do you measure the risk in that third party, and how do you effectively mitigate against it?

      Steven: The third-party risk or the vendor-risk management or how that someone evaluates this — it can only go so far, right? Like, how would you evaluate SolarWinds and the Orion product any differently than you would Microsoft Windows and Defender and how it updates and things like that, right? So there’s limitations to what you can do. I mean, you could audit them, or find out their code-review process and all that stuff — and they could have passed that all with flying colors. Or does your checklist say, “Are you looking for advanced adversaries, you know, injecting themselves into your build process at the highest levels of sophistication and espionage?” But even if they check yes to that, which they might not, they probably aren’t having an effective way or mechanism to do that.

      Sonal: One of the things that Alex Stamos — people tend to over-quote him, but he did have a good tweet about this, which is — “There is no good reason for most enterprise software products to talk to random internet hosts all day. It might be time to move on to an outbound network-permission model for Windows servers, so connections only allowed to domains and signed manifest plus internet as defined in GPO.” Is that the right thing to do? Should people be air-gapping? Like what should people be doing?

      Steven: We deal with sophisticated breaches all the time, and this can even apply for, like, crimeware and other stuff, but that is a recommendation that Volexity has been giving for years and years and years to organizations. And it’s often in an incident that we say, “Hey, your domain controller, for example, doesn’t need to be able to talk to the internet.” There’s obviously exceptions to the rules and everything, but usually those can be defined, especially with next-generation firewalls or modern firewalls — you can define what is actually needed, and allow them to do those things, and not allow them to do anything they’re not explicitly required [to do].

      And that’s a model that is the least privileged, it’s like the least-access type model. That’s a little bit harder, depending on your organization, to enforce for users and workstations where they need to browse the web and do all this stuff. And that’s what content filters and certain restrictions are for. You know, unless you’re into, like, a DOD environment or something where it’s a lot more locked down. But that’s usually accepted in a lot of, like, commercial organizations.

      And a server is where — an attacker, if they’re gonna install malware and do things — usually go for it, because that’s where the supply chain, that’s one of the big areas to get to it. Or those are the machines that are not at home, or requiring a VPN. They’re always on, you know, they don’t get rebooted frequently. That’s where malware gets installed a lot, because it’s something that they can count on, and it’s regular. Being able to prevent that, and limit what those can do — that model, if that had been put in place for organizations with SolarWinds — in this specific instance, it would have mitigated that threat.

      Now, if I start thinking outside the box, and this attacker used DNS — but what if they had done command and control activity, and issued commands, and had done that all over DNS? So, the SolarWinds server talks to its local DNS server, your local DNS server goes out to the internet. If they had modified this malware and actually did all the command and control over DNS, instead of doing it over this connection, that paradigm and that shift would have been a lot more difficult to mitigate.

      But that’s the type of issue and security item we need to think about. You could proactively try to address that, or just say, “Hey that’s a lower likelihood, and I’ll address it if that happens.” But by and large, it’s a best practice with regards to minimal access, specifically for servers connecting to the internet and different resources.

      Joel: It’s funny talking about this, because it’s like the history of the security industry is the history of unreasonable requests.

      I know that a lot of people are jumping up and down talking about, like, don’t let production talk directly to the internet. And if you worked at a bank you know for the last 20 years, that’s been the case, right. Like, highly-regulated industries, and people that have invested heavily on security, have always focused on doing these rather idiosyncratic things that don’t make a lot of sense — but made a lot of sense to people who either come from an incident-response or a deep security background.

      You know, back in the 90s, I remember being involved in strenuous debates about why you need to encrypt traffic moving within your data center. And everyone thought it was the most asinine thing because it’s a private link. You’ve got MPLS, no one’s gonna listen to you — and then Snowden released his documents. And it became really obvious why you want to encrypt your data within your data center.

      So, this is just another example where people have been giving best-practice advice, saying, “Hey, you need to make sure that random servers, random production systems, can’t just talk arbitrarily to the internet.” And the response to that has generally been well, that’s an unreasonable request, that takes a lot of work, I don’t know that we necessarily wanna do it. And, there was never a particularly great reason or piece of evidence to point you to say “well this is why”. So, this is why — why you wanna limit that access. And there’s probably a list of other things that are equally unreasonable requests that security people would ask you to do, and eventually they’re gonna have their “this is why” moment.

      Steven: Something that Joel mentioned earlier, which I think is really important, is — a lot of organizations aren’t doing blocking and tackling. They don’t have two-factor authentication on the remote access to their network. They’re using weak passwords, they’re not patching. They don’t know where their assets even are, and their build process is not secured. They don’t even do code auditing or check-in their code. I mean there’s a lot of low-hanging fruit for most organizations. They haven’t even been able to kind of get into some of the basics.

      But I think a big problem that a lot of organizations — whether that’s a government, commercial organization, or really anyone, whether they’re a small company or these massive companies with huge budgets — a problem that they’re facing is that if you had certain security data, you could immediately and very easily answer, “Did I have a problem?” One, did I run up vulnerable software? Because maybe, you patched. You know, I don’t know. Maybe I never ran, and I skipped a version. If you had all your DNS queries logged and the responses, you would say, did I get a C-name? Did I even call out to that command and control activity? There’re certain logs from the endpoints that SolarWinds has instrumented in these event-log data. If you had been capturing that data, you could answer that question.

      Sonal: Most companies do capture that data, don’t they?

      Steven: It depends. If you went into SMBs and mid-sized businesses, even some large businesses, I would say a lot of them aren’t actually logging or keeping DNS logs. And if they are keeping DNS data, it may not be query-and-response. And event logs — the vast majority of organizations don’t have a centralized and long-running retention policy for event logs. But even if they do, their data retention of how long they were keeping this data did not go back far enough.

      They actually had data — they have data going back 30 days, they have data back 60 days, 90 days — so they’re finding out in December about a breach inside of activity that happened and then potentially initiated in May. And, “Oh, I kept all this great data, but I can only go back three months.” Three months from December, it’s September. And for a breach that happened in June or July, that’s, in some respects, useless. That’s a scary place to be in, to not know if you were compromised, or if you were when it started, or what happened, or where did they go, how did they pivot. It’s a missed opportunity, and probably a bit scary for some of these companies is that I was collecting all the right data, but I didn’t have it for long enough, so I don’t actually know.

      Sonal: Wow.

      Steven: We’re helping a lot of companies right now to see what resources they have. You know, we specialize in memory forensics. We’re acquiring memory from their SolarWinds server, acquiring disk artifacts, or full disk images, you know, any log sources. And we have some stuff that we can potentially go in and say “doesn’t look like it” or “definitely, yes you were.” You know, we see these items that clearly indicate that you got a second-stage breach, and you need to expand this out. But we can’t give anyone, if they’re on limited data, a confirmed clean bill of health.

      Sonal: It’s a little bit like going to the doctor and having, like, maybe a continuous glucose monitor for the last year — but you only have the data for the last three weeks stored. And it’s sort of like, “Okay, here’s what’s happening. I’m getting sick, but I only have the three weeks.” It’s just, like, a really tough thing to figure out.

      Advice for businesses and consumers

      I wanna break this down by advice for big companies — like, large enterprises — advice for small and medium-sized businesses, and advice for consumers. So, let’s start with the big companies, because the best threat actors, they understand the reality of modern enterprise IT. What are pieces of advice — or mindsets, even — that you have to offer for how chief security officers, CEOs, leaders should be thinking about the implications of this for their business?

      Joel: I mean, I’ve spent a lot of my career in big companies, and I think the thing to do right now is to think about strategy. Like, the tactics are great, and there’s gonna be a lot of people chasing a lot of actions over the next days, weeks, months. But I think the strategic view of how an organization wants to think about security — as we start to understand what happened, and how it happened — we’ll consistently see in some organizations that security either wasn’t funded, it wasn’t empowered, it didn’t have a remit to act. It may have been under assault. People often view security as being a cost center, as something that you know contributes to the lack of performance in a business. And that is an attitude that is still quite popular.

      So, I would say that, like, it’s really gonna be about figuring out strategically, where does security sit, what’s the right amount to spend on it, how do you effectively empower it, and then how do you partner and build security into your business so that it’s something that helps enable it, versus something that holds it back.

      Steven: Yeah. Generally, no one really thinks like security is not important. I don’t think we ever hear that. Now, action may speak louder than words sometimes. But I think a lot of people think about, “Oh, it’s an afterthought. I’m gonna add it later,” or “Oh, yeah, yeah, well, you know, we’ll do that one day.”

      And I think, like, our main advice to a lot of these different organizations — whether it’s a startup or a midsize company, a company that’s growing really rapidly – is not necessarily that they need to come out of the gate and have to have every imaginable security product, they need to be auditing all their source code on day one, they need to have everything locked down, and the latest firewalls, and this filter and all these EDR products. But it’s like, think about that stuff. Are you doing the two-factor? Are you lazy, like, “Ah, I don’t need to put, you know, two-factor on my Salesforce account where all my most sensitive contacts and information is in my organization.” Or, “Ahh, I don’t really need to put it on email. It’s like, it’s easier if everyone can just log straight in.” Or, “I’m just gonna share this root, you know, Amazon key to get into AWS, because that’s just how our organization’s growing, and we’re not formal.” There’s things that people can do — best practices, actions that organizations can take — see what you can do now, see what you can do along the way, and put that on your radar, so you’re not in a position where you’re starting from scratch, or trying to investigate a breach, or figure out if you even had a breach. We all knew [what] we should have done, and we knew that two years ago. And we run into that, a lot.

      Sonal: Don’t wait till later. And now advice for advice for consumers, like, just day-to-day people like family members, etc. What would your advice be for how to think about things like this?

      Joel: We wrote a really excellent blog post last year called the “16 Things You Can Do to Protect Yourself,” and I would strongly recommend that people do all of those 16 things. It’s all really basic stuff, and it starts with two-factor authentication, patching your systems, and goes all the way down to how you want to think about securing your potential social media accounts, etc. So.

      Steven: Yeah, we issued some guidance, and it’s a couple of intersections of prevention and detection, and then remediation, if you have an actual threat or concern.

      From the prevention side, prevent unnecessary access from your servers — like, your SolarWind server, other devices — from talking to the internet. That’s a prevention mechanism. You know, monitor your assets, see where they’re logging in from, if you have that centralized logging or like a SIM, same thing. Make sure you’re capturing either from event logging or your endpoint security products or that the actual commands being run on the system are being logged. Because that can be pivotal and be critical to 1) detection — but even if you’re not actively monitoring it, you can go back and say, “Hey, what commands are running on this server that’s not consistent with what our system-admin or the typical activity would do.”

      But take a look at your mail server, look at where your email is going, because that’s where the attackers, I believe — they’re way ahead of the game with regards to the things that they can do in Office 365 and Azure AD, where they are so familiar with the administrative commands and what to do from a sys-admin aspect. They’re able to do a bunch of things and hide in ways that people have never even thought about and encountered. And it’s not necessarily, like, they’re ghosts or they can’t be found, people just don’t know to even look for it.

      And then, just from a general remediation perspective — once a device has been backdoored or compromised, it’s an untrusted system now. Don’t just, like, roll back to an earlier version, or, I’m just gonna upgrade to the new version. We say, hey — blow that whole system away. Start with a fresh, clean install. If you’re putting SolarWinds Orion back on it, download the newest version that’s not backdoored, and start everything from scratch.

      Anything you used on that server, if your SolarWinds set up for the Orion had credentials, change all those passwords, and make sure those passwords aren’t similar to, like, old passwords that were used. You know, another thing, too, is — any sensitive API key integrations and things — like, we saw two-factor bypass to get into email by this threat actor. Because they had taken a secret key, and would generate cookies and skip into the email system while not actually being challenged for two factor.

      You’ve got to think about the stuff that someone could steal if they’re in your network, related to this — but also that advice extends well beyond this threat actor and SolarWinds specifically.

      Sonal: That’s great. I’ll include links to Volexity’s blog posts as well as the “16 Things That You Can Do to Secure Yourself” in the show notes. Bottom-line it for me — what’s your takeaway?

      Joel: It’s consistent with what we’ve been saying for a while now. The hardest problem to solve is third-party risk, and this is probably the most significant third-party breach that we’ve seen in history. And so, I think it’s gonna take us months to really understand what happened, and probably years to fix it.

      Sonal: Thank you so much, you guys, for joining this episode of 16 Minutes, which is a 3X 16 minutes.

      Steven: Definitely, thanks for having me.

      Joel: Yeah, thank you so much. And, Steven, it seems that we’re always catching up when the world is burning down.

      • Steven Adair

      • Joel de la Garza is an operating partner at a16z focused on information security related companies. Prior to joining the firm, he held top security roles at Box, Citigroup, and Deutsche Bank.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Developers as Creatives

      Jeff Lawson, David Ulevitch, and Sonal Chokshi

      The rise of developers — as buyers, as influencers, as a creative class — is a direct result of “software eating the world”, and of key shifts in IT from on-prem to cloud & SaaS to the API economy, where application programming interfaces are essentially building blocks for innovation. Developers therefore not only play an outsized role in high-performing tech companies — but managing and motivating them is actually critical in ALL companies, since every company is a tech company (whether they know it or not).

      As every industry turns digital, and a company’s interface to their customers IS software, “asking” one’s developer is the key to solving business problems and to thriving not just surviving, argues Jeff Lawson, CEO and co-founder of cloud communications platform-as-a-service company Twilio, in his new book, Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century. So in this episode of the a16z Podcast in conversation with Sonal Chokshi and David Ulevitch (who previously argued “the developer’s way” is the future of work), Lawson shares hard-earned lessons learned, mindsets, strategies, and tactics — from “build vs. buy” to “build vs. die”, to the art and science of small teams (“mitosis”) — for leaders and companies of all sizes.

      But what does it mean to truly treat developers as creatives within an organization? What does it mean to be “developer first”? And how does this affect customers, product, go-to-market? All this and more in this episode.

      Show Notes

      • How software has become an essential part of the supply chain [1:40]
      • Discussion of whether companies should build their own software or buy it [4:10]
      • Including developers in the sales process [8:10], seeing them as influencers within a company [14:08], and having a culture of asking developers for input [17:13]
      • Defining the “developer way,” or the culture of work that developers prefer [20:00]
      • The importance of small teams [23:47] and how to keep teams small as a company grows [28:37]
      • Lessons from Twilio’s IPO and expanding the role of sales [31:17]

      Transcript

      The new software supply chain

      Sonal: Since I hate starting with the “why’d you write the book” question, I’d love to instead start with the big picture. We’ve seen 3 eras of software: from on-prem to cloud and SaaS to the API economy; you not only outline these shifts in the book, but you go further and argue that software is the new supply chain. So, I’d love to hear more about how you think about that to start us off.

      Jeff: Yeah, thank you for asking, Sonal. I mean, it’s interesting, I come from Detroit, it’s the automotive capital of the world. I grew up around cars, and so many people that I knew, if they didn’t work directly for the automaker, they worked for a company that supplied the automakers. And it was very easy to understand the very sophisticated supply chain that allowed them to manufacture a complex thing like an automobile: General Motors doesn’t have to manufacture every little piece of that car. There’s companies who specialize in speedometers, and headlights, and seats, and all this kind of stuff — they just pick the vendors that are going to help them get to market as quickly as possible, with the best product.

      And that’s what finally the software industry is developing. Software’s obviously been around in some form or another for 50 years, and in its modern internet-enabled form for close to 30 years. I was a developer starting in the 90s, writing code; back then, you basically had to write it all yourself, and you could pull in some open source stuff here and there. But building software has gotten easier and easier and easier, because of sophisticated APIs, and abstractions, and all this kind of stuff.

      However, the scale of the internet, and building applications at internet scale, has gotten so much harder. And so, if you think about the progress that’s happened over the last decade or so of APIs that run in the cloud — if you need a piece of functionality, whether it is compute storage, payments, communications, maps, you name it — all you need to do is sign up, put in a credit card, and plug in this few lines of code, and now your app is supercharged with these powers… What that really is, is the development of a supply chain for building software.

      Sonal: That’s great. And so, I mean, you’re basically saying software is innovation in that context. And we totally agree, obviously. The obvious next question that begs is, what do you build versus buy? In your book, you had a really neat rule of thumb, which is, that software that faces your customers, you should build; anything where your customers will be saying, why doesn’t it do X, and your answer is, well the thing we bought doesn’t do X, et cetera, et cetera — You basically argue that you can’t buy differentiation, you can only build it.

      So, talk to me a little bit more about that, and really, where do people then compete? Because if everyone has access to the same APIs, like, where’s the differentiation?

      Whether to “build or buy?”

      Jeff: Absolutely. You know, something happened over the last 15 years, which was software went from the back office to the front office. It went from being something customers don’t care about to something they experience every day.

      David: The remote control in our pocket — it’s how we interface with the companies we do business with!

      Jeff: You know what I called my iPhone for a while, I called it my “summoner”.

      David: Yeah, exactly.

      Jeff: Think about your bank. Twenty years ago, your bank was a storefront that you walked into; it was clean, the teller was friendly, and they gave your kid a lollipop — okay, I like my bank. And now your bank, of course, is a mobile app. Suddenly, the interface you put in front of that customer is the perception of your product and of your value as a company. You like your bank if the mobile app is fast; if it is bug free; and if it has a lot of features and functionality to make your life a little bit easier.

      Back in the days when it was back office, it would be common for IT departments to say, “Okay, should we build versus buy?” And a vendor would inevitably come in and say, “Don’t reinvent the wheel”, and you just bought something off the shelf. But now in a world where the software you use is your source of competitive differentiation, the act of building is the act of listening to your customer — and so now, the question has gone from build versus buy to build versus die!

      Because one participant in that market starts listening to customers, and using the agility of software to innovate faster and faster and faster. And then the incumbents in that industry start listening, and they say, “Oh wow, we’ve got to do the same thing” and so they start becoming builders of software as well — this Darwinian evolution is going on in every industry. And so the buy act is one enabling you to be the best builder in your industry. That’s how I think about it.

      David: That is great.

      So, you know, if you are a company, you’re thinking about, “Hey, I do want to start building out developers; I want to have teams. But do I just use all the AWS kind of APIs? Do I go to Microsoft and use Azure?” There’s all these new startups that have APIs, how do I think about those things, how do I make those decisions?

      It seems like AWS is creating an equal API for every startup that’s out there… how do you talk to business executives about those decisions?

      Jeff: Well, it’s like any competitive dynamic in any supply chain. Which is, you want to pick the vendors and the partners who are going to enable you to go build as fast as you can. Every company has their areas of strength, their areas of expansion. And APIs offer you the ability to pick the services that are going to serve you best.

      And one of the things I talk about in the book, is actually trusting your developers to help you navigate the vendor choices that you have. I remember back to the early days, when I was a developer, I would do a Google search; oh, this looks good; okay, how do I get started? And you click and it would say, “Contact sales. And, you know, if you sign an NDA, you can read the documentation.” You’d be like, okay, back, never mind, I don’t want that.

      Because for a developer, documentation is the ultimate marketing. Yes, every company has a marketing website that’s pretty and hand-wavy. But at the end of the day, the documentation of an API is the perfect description of what that product does — it literally describes every in and out of what the product does. And you don’t have to believe a salesperson, you don’t have to look at a slide deck; which is like faster than in the old world, you couldn’t even get a meeting with the salesperson, right?

      That’s why they are becoming so influential in adoption cycles and sales cycles, because a developer can — for free — read the documentation, make an evaluation of what they think the product is. And then for literally dollars, build the prototype, and test those hypotheses, and put it in front of customers, and actually do a beta. And that is a completely revolutionary way to de-risk these projects and take them from a bunch of hypotheticals — with a lot of budget and energy put into signing contracts with vendors and taking meetings — to actually just getting hands on a keyboard, building the thing, and putting it in front of customers.

      David: That empowered developer really transforms the go-to-market for API companies, right? I mean it changes the way you do customer success, and the way you do onboarding, if they’re going to build that prototype before they maybe even talk to you — it must really radically transform the way you think about what an enterprise go-to-market organization looks like in an API world.

      Seeing developers as customers

      Jeff: I’ll tell you a true story; WhatsApp is a very large customer of Twilio and has been for a long time. And like, literally, that is a Yahoo email address — of Jan, signing up for a Twilio account back in 2011 or ’12, or whenever it was — and this is one of the big differences between API-economy companies, and other companies, who say they serve developers. At other companies — who regularly launch APIs and say hey, we’ve got a platform — the developers are a strategy. At API economy companies, developers aren’t a strategy… They’re our customer. They are our revenue. You’re never going to pull the rug out from under them, because you are dependent on them for the health of your company. And that’s a very different world than other companies where the customer is an advertiser or somebody else; for the API economy, you have to treat the developer as your customer.

      David: Yeah, the example of the WhatsApp story of having an individual developer sign up, means that you really rethink marketing, communications, how you engage with those customers, how you measure the metrics, how they’re using the product.

      Lots of companies don’t have great visibility into how their customers are using their product. But by definition, an API company has incredible visibility into how their product’s being used. And that has never been possible before. You know the nice thing about an API company is you don’t have to track: Did they build a prototype? Are they going into production? Are they making one or two calls a week, are they now making thousands of calls a week? Maybe we should reach out, see what they need, what features are missing, have a product manager engage them, and you know, keep those people close to the customer, whether it’s developers or product managers.

      And so the order of operations of the traditional enterprise go-to-market HAS shifted. What used to be a whole bunch of pre-sales, marketing material, and brochures, and websites; now, as Jeff said, it’s the documentation. But then after that, you do want to come in with that white-glove kind of a service and really embrace your customer, understand what their needs are, understand what the opportunities are; you know, maybe rethink your roadmap and all these things, based off of how people are using the products.

      And I think that creates lots of other opportunities for startups to actually support this new kind of a go-to-market motion.

      Sonal: I think the most under-discussed, but most important aspects of this conversation IS this notion of keeping developers close to customers. That’s a really novel idea for a lot of traditional companies; it’s actually probably even a novel idea for a lot of established software companies, frankly.

      You both mentioned the documentation. But what really struck me, is it forces developers to be better communicators. Because you’re essentially having to explain (even if you don’t write all your own documentation), what is this value, what is this thing you’re doing? And that is another segue to this topic of how does one keep developers close to customers? Does that mean you literally tactically put them in front of the customer; are they now the front interface to customers? Are they taking the customer success calls? Are they taking, you know, reports?

      Like, what does it actually mean to keep your developers close to customers; and, how should this happen (or not happen)?

      Jeff: That’s a great question, Sonal. I think the answer starts with my assertion that being a developer is fundamentally a creative exercise; it’s not merely a technical exercise.

      And I think that’s something that is really misunderstood about software developers. You know there’s this pop culture myth about developers that’s propagated by Hollywood; and, look, there may be some truth to that, but, really, developers are not just, like, calculus, you know, math nerds. In fact, we did a survey of software developers, and we found more than half of them played a musical instrument, and it was like three quarters of them did some sort of artistic thing on the side. And the act of writing software is creative problem-solving.

      But that creative problem-solving skill doesn’t end with writing an algorithm — it really goes all the way to the types of problems that you throw at developers. And so one of my biggest statements in the book is, instead of sharing solutions with developers, share problems. Instead of handing a product-requirements document that was written by some MBAs, and throwing it over the wall, and build it to the spec — you know, having a developer basically be a digital assembly line worker — share the problem with them: Hey, we’re trying to make it so customers can sign up for our product and get productive in 30 seconds instead of the 20 minutes which it takes today. NOW you unlock the ability for that developer to use the full creative energy they have.

      David: You and I both self-identify still as software developers; I still write code and I’m sure you write code as well. The reality is, as you said, these developers are creatives — and like any real creative, they want people to use their work, their art.

      I actually believe that there’s like a selfish reason why people are open source developers, which is that they just get a much wider audience much more quickly. And then inside of a company, you want to know that there are people that are going to pay tens of thousands, or hundreds of thousands, or millions of dollars to use the code that they wrote — and, find that extremely satisfying. One of the greatest tropes that always bothered me was this idea that developers need to be protected from the customer.

      Jeff: Don’t get me wrong; you don’t want your software developers handling every support ticket and every sales cycle. However, if you don’t poke holes in those siloed walls — and you treat them like these precious things that can’t be bothered by such trivial matters, like customers – well then you are doing a huge disservice, ‘cause you’re essentially blinding the developers to why they’re writing the software in the first place.

      And so you need to intentionally poke holes in those walls, and I think product managers are actually the key to this. At a lot of companies, product managers see their jobs as shielding developers; and I think the job of product managers is to figure out how do I facilitate the right interactions between the developers — who I want to be able to have instinctive understandings of my customers and their problems, and the jobs-to-be-done by those customers — and the development team who’s there to solve problems. Because when you have an instinctive understanding of the customer, well so many other ways of solving problems arise, and so many other ways of thinking.

      Developers as internal influencers

      Sonal: It’s super important to treat your creative class that way. And it’s so funny because we also talk about the rise of design a lot; and this is a similar shift that’s happening with designers when it comes to designing technology products as well.

      I’ve noticed that people often do the same thing with writers and editors. Like they give you this specs doc, and I’m just bring ‘em more upstream, like, embed into your flow… ‘cause we’re going to hear things that you don’t know to ask us or tell us.

      Jeff: Sonal, one thing that comes to mind is the parallel between the shift that’s happened because of personal computers and the internet for other creative classes — we’re all aware of the fact that you can use GarageBand, or Pro Tools, and a musician in their own home can record a song with basically the same tooling that the professionals use. And if your music is any good, you can develop an audience of millions of people, as a creative. The same thing for film production or video, right, like you used to have gatekeepers, who were studios, and you needed millions of dollars of equipment to make a movie; now, anyone with an SLR can make a movie, can edit it on Final Cut Pro (the same software that they use in Hollywood), and upload it to YouTube.

      And so people well understand what’s happened to those creative disciplines. But really, the same exact thing has happened for software developers. Which is a software developer can take the same infrastructure that’s used by the largest companies in the world; can build a software app on the internet; and get distribution with Google AdWords, or Facebook Ads, or any of the stuff. And a developer with the right idea is also liberated to be able to build just about anything they need — in that exact same way that musicians or video artists, or storytellers do. And that’s an amazing thing that’s happened.

      Sonal: Combining that with what David said about open source, it does create this sort of composability — build on top of each others’ building blocks. I mean, the best thing about TikTok is remix culture; like the fact that you can remix all these bits. And that’s exactly the same thing you’re talking about.

      You know this notion of developers want an audience, developers are a creative class — what does it mean for developers to become influencers more broadly within a company; with the question being, how to make developers more influencers across the company?

      Jeff: Well to me, really, that comes down to giving developers a voice. And an environment where you embrace experimentation — experimentation is the prerequisite to all innovation. You enable THAT as opposed to more hierarchical, top-down, highest-paid- person’s-ideas wins, and all that kind of stuff.

      David: You know Jeff, so many companies have not embraced the Ask Your Developer mindset. Sometimes what they do is they sort of find their way into the shallow end of the pool by… sort of having hackathons. And then magically, they find out that really good ideas come out of these hackathons. And hey wait maybe, maybe we should involve the developers earlier in that product-roadmap process.

      You know they have these good ideas, but they never get prioritized, they never get surfaced; like you said, they come from elsewhere in the organization, but maybe they shouldn’t. How do you think about hackathons? You know, when you’re talking to those business executives, how should they think about hackathons? And then how can they take that catalyst of sort of an event inside the organization and actually institutionalize that into their culture and workflow and process?

      Jeff: You know, I like hackathons, not necessarily because every hackathon results in the next giant innovation or whatever it is — you’re right, it often does end up proving the hypothesis that, oh wow, there are some things that we could do relatively quickly, that are very impactful, if we let our teams kind of go wild thinking about what are the things — but I like hackathons because they are a practice that actually encodes Ask Your Developer.

      ‘Cause if you think about it, inherent in a hackathon is this idea of letting developers essentially spend a period of time self-organizing and building the things that they think are interesting and important, and using that opportunity to prove out and to test out their ideas. In an ideal world, companies would operate more normally, in more hackathon-oriented ways — i.e., small teams working iteratively, and being agile, and being tasked with problems not solutions. And a hackathon is a way to simulate that, for a short period of time, and at small scale.

      Sonal: I mean, I hear what you’re saying, but I feel like hackathons are a bit performative. Because I’ve seen too many times, like a lot of companies do what you describe in the book as that “Silicon safari” effect, like animals in a cage; we must follow the same practices, and perform them essentially.

      David: I don’t think they’re performative. I think that they’re like, you have pressure building up in a system, and it’s like the steam valve — you reconfigure the machinery so that that steam valve doesn’t need the release.

      I don’t think there’s ever been an organization (at least that I’ve ever heard of) that’s done a hackathon. And been like wow, that was totally useless, we’re never going to do that again. They may not get that great new product that, you know, sends up their revenue for the next five years. But there’s always learnings — and those learnings are not just in the code that gets written, but in the processes that get created. So, I think hackathons are great. I would certainly not describe them as performative.

      Jeff: I will play the role of peacemaker here, because I think you’re both partially right. I think that, look, if you go into a hackathon saying okay I really am waiting for these folks to come up with the thing that’s going to save the company — it’s probably not really the right expectations to walk in with. So, to some extent, it is performative.

      But I think that the goal of the hackathon is not to solve the problem during the hackathon. The goal of the hackathon is actually to model what you want your organization to become; it’s like a rehearsal for really, the organizational structure and the way of operating during the regular course of business. And so I think that’s the role that hackathons play.

      I actually think a better way to structure it is, if you’re an executive at a company, create a two days a week, whatever you want to do; but go in with, hey, I care about this. You’re important, I’m committed to this. #1.

      #2: here is a list of the 10 biggest problems I hear from our customers; or here are the 10 biggest problems that we face as a company — and I’d love for you to be thinking about. NOW you’ve directed the energy, you’ve shared problems with those developers — and you’ve told them the stakes are high… I think that is a much more effective way to run a hackathon.

      Sonal: I love that. You’ve made peace.

      Developers’ unique workflow

      We’ve been talking about the developer mindset quite a bit. But we haven’t actually defined what is the developer way here: It’s not just a role and a function; like, it’s a mindset. And Jeff, have you seen in your work that these habits transfer across the org? You use the word “mindset” throughout your book; and David has used the word “way” throughout his work.

      I would love to hear your guys thoughts, kind of define what makes a developer.

      David: Look, I think developers in general — especially open-source developers — have mastered a whole bunch of working methodologies that end up just turning out to be great working methodologies not just for developers, but for anybody.

      So, that involves really having the tools to do asynchronous sort of communication and development; so in software development, that could be revision-control systems, things like GitHub, or GitLab, which allow people to collaborate. It can be ways of memorializing decisions: developers have change logs, they have issue tracking, they have pull requests — and so it’s often very easy to figure out how did this line of code get into the codebase; who signed off on that decision; who else reviewed it? And these are things that other organizations (outside the developer part of the organization), no one knows how the decisions get made; who made those decisions; when were they made; why were they made?

      And then, of course, there are power users of their own computing devices. And so, you know, developers often are much more keyboard-driven, they use shortcuts, they’re much more fast to operate. And we see these things bleeding into our world today; people now use emojis as shorthand. People are now using things like a command palette, and they’ve gone way beyond the way developers use command palettes; they’re now sort of bleeding into our normal daily life.

      But I do think there’s a lot to learn from the way developers organize, the way they communicate, the way that they memorialize decision making. And then, of course, the way they just use their computing tools as power users, because, you know, everyone’s effectively a digital native these days and becoming more and more of a power user.

      Sonal: How do you define it? Curious for your thoughts on this.

      Jeff: You know, I’m a little more hesitant to define like, the developer way. I struggle a little bit with saying, you know “here’s my definition of developers”, because there’s a lot of different ways to work.

      Now that said, a lot of developers do share a lot of common traits. Like when your work involves writing Boolean logic, a lot, if you tend to be drawn to that work, you probably tend to also want to have logical thinking in other areas of life. So I do tend to see engineers as being logical thinkers. And, you know it’s interesting, because like I, as a CEO (and a software developer), bring logic to a lot of the decisions, but also to a lot of my interactions with other team members.

      And I actually have noticed that it can be rather infuriating, actually — it’s one of things I’ve had to moderate as being a CEO, from being a developer — I’ve actually realized some of the ways in which the ways developers think, while they may be often right, they don’t necessarily serve you in interfacing with people who don’t think the same way.

      But I would say, if you’re a business executive, a few things to think about: One is, like many other arenas, where you have a lot of concentration required to do your job, *flow* is one of the most important things for developers — so the ability to immerse yourself in a problem, be able to kind of fit it all into the working memory of your brain, and then be able to get your work done is really important. That’s why developers are really sensitive to interruptions, to taps on the shoulders, or meetings, and things like that.

      And the other thing I would say is, if a developer is poking holes in the logic of your idea or your plan, they’re not being a jerk; it’s just the way they think. They’re processing whatever they’re hearing through the lens of how they think, and therefore, that’s the response you’re getting. And so, it’s maybe a way for folks to understand developers — and therefore be able to engage with them — is to think about the ways in which developers process information and make decisions.

      I like to propagate this idea that developers are creative problem solvers, and much bigger, more influential parts of the team, when they’re whole human beings. Not just like, you know, code monkeys.

      The importance of small teams

      Sonal: Jeff, you’ve alluded to this a few times — in fact, I thought this was one of the most interesting themes in your book — is, you asserted throughout it’s about small teams, it’s about small teams; it felt like a refrain.

      I’ve always heard the two-pizza rule for Amazon; I never heard the origin story until your book — and you describe having a dozen bagel team — so tell us a little bit about small teams, why they matter, how to grow them, how to make them work? I feel like the title of the book should also be, “Small Teams”!

      Jeff: Yes, “Ask Your Developer: Small Teams Are Right for You.” <Sonal laughs>

      So, back to, we were starting Twilio — at the very beginning, in this very small team that you are, you kind of do everything; everything from like having talked to customers that day, to handled support tickets, to writing code, to understanding the architecture of everything that’s going on. Like, you can hold the whole business in your head at the scale of several people.

      And as we started growing Twilio, one of the most momentous things that happened to me was I was talking to my friend — his name’s Dave Schappell (not the comedian, different Dave Chappelle) — he was actually the person who hired me at Amazon. And he had started at Amazon in, I think, ’97, so when the company was about 100 people. I joined in 2004, Amazon was about 5000 people. And Dave, he quit; that was my first week, he was like, “I’m sorry, I couldn’t tell you before. I’m out of here.” He went and started a company called TeachStreet, that was acqui-hired back into Amazon about seven years later. So, Dave found himself back at Amazon, but now the company was 75000 people.

      So, Dave saw Amazon at 100 people, 5000 people, and then again at 75000 people. And so, as I was starting to scale Twilio’s culture, and thinking about how we were going to structure ourselves, I called Dave and I said “Hey, Dave, can you compare and contrast Amazon at 100 people, Amazon at 5000 people, Amazon at 75000 people?” And he said, “Hunh, let me think about that for a second.” And he said, “You know what? It’s exactly the same. It’s the same bounce in people’s step, the same sense of urgency, the same intellect that everybody here has. It feels like the same company.”

      And to me, THAT is the outcome of the two-pizza team, as they call it at Amazon. Because as the company is growing, there’s a natural tendency for every company, as they get bigger, to slow down, to insert more bureaucracy, to create walls between customers, to create politics and things like that.

      And what small teams do is they keep a small group of people who are very tight, and focused on — what my definition is, the small team is defined by — a customer they’re serving, a problem they’re solving for that customer, and then metrics of success that say whether or not they’re succeeding. And there’s a lot of advantages here:

      First of all, on a very small team of say 10 people, there’s no room for a low performer to hide; on a team of 10 people, everyone’s got to carry their weight, and it’s obvious when somebody isn’t. The other thing that I think is interesting about small teams, is that people’s willingness to go along with decisions is proportional to how involved they were in that decision, and how close they are to the decision maker. And so if you’re on a small team of 10 people, and there’s a single-threaded leader to lead that team, then you want to push as many decisions as possible to that leader. And when you do, it’s likely that they’re going to be involved in that decision, and if the person’s managers would have made a decision that maybe they disagree with, they’re probably going to be more inclined to disagree and commit. Or, they’re more likely to be able to question it; hey, can you explain to me why you made this decision? Like you can’t do that when it’s someone five levels up; usually you don’t even know the person, or it’d be hard to get the meeting, or you’d be afraid to express yourself. And even when there’s disagreement, those disagreements get resolved. So instead of having this like us vs. them, you get this sense of: okay, you know we’re all on the same team here; you know let’s go, let’s do this.

      David: You know, one additional benefit of small teams that I’ve always observed is that there are some people that like to work on very small projects with rapid iteration, where they sort of have the dopamine rush of shipping a release and getting something done very quickly. There’s also other kinds of engineers that like really loooong, hard projects, that take months and months and have very little to show for it for a long period of time. And by having small teams, you can actually let people sort of work in an environment that works best for them. Like those people that want to close out a ticket to help win a deal, or save a renewal — like those people like to be on this fast, close-to-the-customer kind of teams; and there’s those infrastructure people.

      And, by having small teams, you allow people to end up gravitating towards the kind of work that ends up allowing them to work at their highest and best sort of potential. People can find where they fit in best, and I’ve always found that as an organization scales, to be a really, really valuable component.

      Jeff: The other interesting thing by the way, about the infrastructure people you mentioned — great engineers love building for the other builders, right? — but they’ve got to see it as I’m serving a customer with a mission and metrics.

      And so, even internally focused teams, it works the exact same way. And I think that’s one of the beautiful things about structuring yourself that way, is reminding everybody: Like, if a team exists and has no customer, internal or external, then, man, I’d wonder why they exist.

      Managing growth and team size

      Sonal: I want to probe into like, what happens when companies scale and grow, and, small teams can’t really stay small — You argue for a really interesting concept called “mitosis”, which obviously is borrowed from cells, that split as you grow. And I thought that was a really interesting idea.

      Jeff: Yes. So for us — I’ll give [an] example — our first product was Twilio Voice, the ability to make and receive phone calls with Twilio. And you know that was built by the founding team, we built it, we started hiring people, we grew. And suddenly the team that was working on that product became like 15 people. And we said okay, this is getting too big. If we believe in small teams, we need to split this. How are we going to do that?

      And so what you do is you take the problem domain, and you say okay, if I want to divide this problem domain in half — so I can have two teams instead of one big team — how would I do it? And there’s no one answer to it, but the best thing you can do is align the people, the technology, the code itself, and the customers. And when you can figure out how to actually divide the problem so that the customer, the technology, and the team can actually stay together, that’s the best way to do it.

      And so for like our voice product, we realized that the voice product really consisted of two things: One was the connectivity layer into the carriers of the world; and the second was all the programmable APIs that allowed you to do things with that connectivity. And so we divided those two teams. And initially, the code was completely intertwined, and it was like a complete mess. And we sat out and we said, okay, we need to decouple those two systems; and we need a technical leader for the connectivity side, we need a technical leader for the API side. And… over the course of about six months, we untangled the code bases, we untangled the teams. If we didn’t have a leader we needed for the next team, we would hire the leader. And after about six months, we were able to decouple the two, and take one very big team and turn it into two small teams again.

      And that’s basically the process that Twilio has used to grow from, you know the three engineers that we were when we founded the company, to now… several thousand engineers. We just keep doing this mitosis process.

      In the act of that, one of the key enablers of that is itself, APIs — and those APIs can be used internally, but they could also be exposed externally (if we wanted). And so we actually ended up doing that. We productized — we call it SIP trunking — that’s the connectivity layer, that is now its own product; and that product itself has undergone mitosis now many times, as well as our API layer, which is its own product.

      Do you throw away the notion of small team and say well, that’s only for the early stages, once it gets big, so be it. I think that’s exactly the wrong answer.

      Sonal: That’s fantastic, and I have to say, people really should read your book, because you say a lot more about the types of leaders that are needed, and I love that you have this line about — a phrase that you’ve coined — “The fallacy of better collaboration” — because that’s one of my pet peeves — where when you have too many small teams, how do you coordinate and collaborate? And it’s a wonderful, wonderful chapter.

      Lessons from Twilio’s IPO

      Last question. One thing that I’ve been dying to ask you, just super quick, which is, what would you say is your biggest personal evolution, pre- and post-IPO? That’s top of mind for a lot of people right now, so I’m very curious about that.

      Jeff: For me, it has been — the biggest evolution has been — really thinking more holistically about the intersection of product and go-to-market.

      You know, we went public, and we had about 12 sales people in the company. And we really loved our developer-first approach, our self-service model; developers sign up and start building. And… you know, we were very happy with that, and as such, like really had under-invested in sales. You know like 12 sales reps to manage a quarter billion in revenue, that’s an underinvestment, right?

      Sonal: Wow… yeah!

      Jeff: But what I came to realize was, empowering a developer to get started with Twilio is amazing. But once a company starts spending hundreds of thousands or millions of dollars, you can’t rely on a relationship with a developer to maintain that level of spend, because now you’ve got so many more stakeholders inside of the company.

      Developers want to do great work, but when the CFO is saying, “Hey, how come we’re spending this much on Twilio?” like “I don’t know, I… ” I mean, you know, that someone else’s job, right? And so we now we call it the “developer-first” approach, where, developers start the relationship, but then we build a mature relationship with many stakeholders inside the company — and that’s essentially what salespeople often do, is they understand the org chart of the company; and they understand who the stakeholders are; and they really build deep relationships with the customer (the customer, meaning the company, not just the individual.)

      You know so that’s probably I think one of the biggest things that I’ve come to you know, evolve, in my thinking is the holistic nature of what it takes to build a company.

      Sonal: It’s so funny, we have a whole series of podcasts called How to Go from a Technical to Product to Sales to Go to Market CEO, because it’s exactly the journey.

      That’s fantastic, Jeff Lawson, author, CEO of Twilio, and author of Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century. Thank you so much for joining!

      Jeff: Thank you, Sonal, it’s been a pleasure.

      • Jeff Lawson

      • David Ulevitch is a general partner at a16z where he invests in enterprise and SaaS companies. Prior to joining the firm, he was the founder and CEO of OpenDNS (acquired by Cisco).

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Machine that Made the Vaccine

      Stephane Bancel, Jorge Conde, and Hanne Winarsky

      In this special episode, which originally aired the day the FDA authorized the world’s first mRNA vaccine for emergency use, Moderna CEO Stephane Bancel tells the story of the machine that made the vaccine: the platform, the technology, and the moves behind the vaccine’s development.

      This episode of Bio Eats World takes us from a world of pipette and lab benches to a world of industrial robots making medicines: We used to grow our vaccines, now we can “print” them, getting them to patients faster and more efficiently than ever before. In conversation with a16z general partner Jorge Conde and Bio Eats World host Hanne Winarsky, Bancel describes the exact moment he realized they might actually be able to make a vaccine for COVID-19; what happened next to go from pathogen to design; how this new technology that uses mRNA works (in a chocolate mousse metaphor!), and what makes it different from “old” vaccines; and how to think about managing both innovation and speed in this world. Why is this such a fundamental shift in the world of drug development? And where will this technology go next?

      Show Notes

      • What happened when the SARS-CoV-2 virus was discovered, and Moderna’s response [1:44]
      • How Moderna’s vaccine was developed digitally [4:02]
      • Description of how mRNA drugs work [5:43] and the history of this technology [7:30]
      • The advantages of mRNA drugs [11:18]
      • How Moderna learned about the positive results from Phase 3 trials [14:41]
      • Why mRNA drugs can be produced more rapidly than other processes [19:06]
      • How mRNA technology might be used for other diseases, and where it is limited [22:18]
      • Details around Moderna’s manufacturing process [27:44]
      • How Moderna was founded, their original goals, and challenges faced [32:01]

      Transcript

      Hi, I’m Lauren.

      Hanne: And I’m Hanne, and this is “Bio Eats World”, our show where we talk about all the ways biology is technology.

      Lauren: This week, in place of “Journal club,” we have a very special episode featuring Stéphane Bancel, the CEO of Moderna, in conversation with you, Hanne, and a16z general partner Jorge Conde. And we’re talking about the COVID-19 vaccine, right?

      Hanne: Yep, that’s exactly right. The conversation is a really incredible dive into how they developed one of the world’s most awaited vaccines. Bancel describes everything from the moment he first realized they could make a COVID-19 vaccine with their technology to the day he heard the first data on how effective it was in humans. In this episode, which is airing just after the Vaccines and Related Biological Products Advisory Committee meeting makes its recommendation to the FDA, Stéphane tells the story of not just how the vaccine got made, but everything about the machine behind this vaccine — the fundamentally new platform and mRNA technology behind the vaccine’s development.

      Lauren: This vaccine is really one of the first medicines that is part of a bigger transformation from a world of pipettes and lab benches to a world of industrialized machines making medicines. We used to grow our vaccines, but now we can print them — getting them to patients faster and more efficiently than ever.

      Hanne: Bancel describes what it took to go from pathogen to design to clinical grade product; how mRNA works, in a chocolate mousse metaphor; and what makes it different from old vaccine technology — why exactly this is such a transformative shift in the world of drug development, and where this technology will go next.

      The discovery of SARS-CoV-2

      Jorge: Stéphane, sitting here now, in December of 2020, could you imagine a year ago that mRNA as a concept would be a household name?

      Stéphane: No, and we have a lot of things going on in vaccines, in cancer, in autoimmune disease, in cardiology, in viral genetic disease. But I had no idea that 2020 was going to look that way.

      Jorge: So, if we flash back to January of 2020, can you talk a little bit about the process that you went through to realize that you potentially had the technology that could be a solution for this emerging pandemic?

      Stéphane: Yes, so I’ve been working in infectious diseases all my career, and I’ve developed an eye for outbreaks. So, one of the things I do is I read the Wall Street Journal and the Financial Times every morning as I get up. And between Christmas and New Year of last year, I noticed an article saying that there is a new pathogen agent in China giving pneumonia-like symptoms. That’s all it says. And so, I sent an email to somebody working for Tony Fauci — Barney Graham, who we’ve been collaborating with for years designing several vaccines together. 

      And I said, “Barney, have you seen the new pathogen in China? What is it? Is it a bacteria or is it a virus?” And he replied to me a few hours later, and he says, “It’s not a bacteria. It seems to be a virus, but we don’t know which one yet.” And a day or two after, Barney sent me an email and said, “Hey, we learned from our contacts in China, it’s not flu, it’s not RSV. We don’t know what it is yet.” And then another day goes by and he says, “It’s a coronavirus, but it’s not SARS and it’s not MERS. It’s a new coronavirus. Within a day or two the sequence should be put online by the Chinese.”

      And so on January 11, the Chinese put the sequence online. And our team at Moderna used the sequence to design a vaccine. In parallel, Barney’s team does the same thing. And when they shared notes after around 48 hours, they had designed exactly the same vaccine.

      Creating Moderna’s vaccine digitally

      Jorge: A couple of things that are fascinating about this — number one, the fact that the digital copy of this virus came from China before the biological version reached our shores. That’s remarkable in and of itself, that we knew what we were dealing with, at least digitally, in a matter of days thanks to all of the advances with genomic sequencing technology. But the other remarkable advancement in technology here is what you just described — you were able to design a vaccine based on the digital version of the virus, also in a matter of days, is it?

      Stéphane: So, you’re right, Jorge. And this is the piece that I think most people in pharma don’t appreciate yet — the power of modern technology is, in 48 hours we designed and locked down the entire chemical structure of a vaccine.

      Hanne: Unbelievable.

      Stéphane: And we click “order” on the computer — so it all happened in silico, we never had access to a physical virus. And we designed the vaccine. And with, again, the two teams at NIH and Moderna, because we were so worried — to make a mistake in the vaccine design, as you can imagine.

      Jorge: Of course.

      Stéphane: So, we were super happy when the team literally compared notes after two days and they had exactly the same design for a vaccine, because it was an outbreak and we knew every day mattered. At the same time, we started to make clinical grade products to go into a Phase 1. And that is really remarkable — is the vaccine that is reviewed by the FDA on December 17, it’s exactly the same vaccine that our guys designed in January in silico. We never changed one atom. It’s exactly the same molecule.

      Hanne: So, it’s the same vaccine that took 48 hours to design.

      Stéphane: That’s going to help hundreds of millions of people next year, yeah.

      How mRNA vaccines work

      Hanne: Can we take a moment to just get really simple and talk about how you would define this messenger RNA technology, and what you wish the public understood about how mRNA works?

      Stéphane: Right. Yeah, so it’s a molecule that exists in every one of your cells that is basically the xerox copy of an instruction of your genome for one gene at a time to make protein in your cells. So the way I would describe it to my two young daughters is — think about DNA like the hard drive of life, where all the instructions of all your 22,000-ish genes are stored. And think about it a bit like this is a recipe book that your grandma gave to you before she passed away. All your favorite recipes, that’s the hard drive, that’s DNA. And when you want to make, let’s say, a chocolate mousse, if you go with your grandma’s precious book into the kitchen, you’re going to damage your book a lot. There’s going to be flour, and eggs, and sugar. And after a few times, you might not be able to read the recipe anymore. So what evolution has done, which is beautiful, is to protect the integrity of the instruction in the hard drive, in the book. When the cells want to make one protein, like let’s say insulin, what it does — it makes a copy of the instruction only of insulin in the book, like my example of chocolate mousse — a xerox copy — and takes it into the kitchen (i.e., the cell) to make for a little machine called the ribosome, that I describe [to] my kids as a little 3D printer that reads the message with the instruction of mRNA, and makes the protein by adding one amino acid at a time. So it’s a natural molecule that basically carries genetic information to make proteins.

      History of mRNA technology

      Hanne: But using mRNA as a tool in the way you’ve been doing it, that was not always an obvious approach. So can you talk a little bit about where that began, that idea, and what it first looked like?

      Stéphane: Yeah, so it’s actually very interesting. When mRNA and DNA were discovered, actually people in a lot of universities tried to make medicines out of mRNA because it was a very logical use of mRNA. Just copy nature — make a synthetic mRNA, inject it into animals before humans, and it should make a protein. Because of what was known about science at the time, including immunology — all the analytical tools that did not exist as part of manufacturing purity and so on — when they would inject mRNA in animals, animals would have flu-like symptoms — a fever, vomiting, diarrhea — because mRNA, if you remember, most viruses in life including COVID-19 is made of mRNA. And so through evolution, mammals have developed mechanisms to recognize foreign mRNAs. And of course, when you inject mRNA as an ID for a drug, that’ll be a foreign mRNA. And so actually people abandoned and just quit on trying to make mRNA as a drug. What happened in the 2010, 2011 timeframe, here in Boston, is you had a set of academics at Harvard and MIT who started to play with mRNA again because there [had] been some new discoveries made in the immune system — that they believed at the time that if you modified uridine, which is one of the four letters of mRNA, you can make an mRNA that’s immuno-silent.

      Jorge: In some ways, when you think about it, Moderna doesn’t make therapies, right? You make instructions that the cell uses to make its own therapies.

      Stéphane: Yeah, correct. We don’t give you the vaccine. We give you an instruction for the cells of healthy people, in that case — to read the instruction, to make one protein of a virus, to make it as well as if they had been infected by the real virus, to show it to the immune system so the immune system can make a neutralizing antibody and mature it. So that if, later, they get infected by the real virus, the immune system is ready to prevent the virus from [replicating] in their body and getting them sick. But what gets people sick in infectious disease is you have too many copies of a virus.

      Jorge: Yeah, and so I think this is a remarkable thing for a couple of reasons. What you’re essentially doing is you’ve looked at the virus’s, you know, genome, and you’ve said, “Okay, if I take certain pieces of code from this virus and encode them in mRNA and deliver them to human cells, I am basically giving the human cells the instructions to make pieces of the virus that the immune system will train itself on, recognize, and eventually neutralize.” And in this particular case, that target was the spike protein in the SARS-CoV-2 virus. Is that accurate?

      Stéphane: It sounds correct, Jorge. And the reason that mRNA, in my opinion, is so powerful is that you totally mimic to a human cell the natural biology of an infection without giving the virus at any time. We never give a virus to people — we give, as you said, a piece of the virus. In the case of corona, because it’s a pretty simple virus, we believed — and the clinical data have shown in the Phase 3 that we were correct — that one protein of a virus — a very important one, the spike protein — if you were able to get a high quality of a high quantity of neutralizing antibodies, you should be protecting people if they become infected with the real virus.

      Advantages of mRNA vaccines

      Hanne: Why is it better to have the cell mimic this natural process than in the old technology?

      Stéphane: And that’s a piece that is really unique, because when you think about it, when you get an infection by an mRNA virus in your body, what happens? The virus of mRNA gets in your cell, you use your own cell machinery to make the protein — to basically self replicate inside your cell — and then it escapes your cell. And this is what your immune system sees. And so if you think about it, the old technology of vaccines, where you’re making an E. coli cell or CHO cell — a protein that then you inject in a human, and that just circulates in your blood. That is not mimicking the natural biology. 

      In our case, the spike protein — we designed the vaccine, so it’s made inside the cell. So in a human cell, not an E. coli cell, and then we design it to be transmembraning — to stay attached to the cell, and to be presented to the immune system that basically backfalls, you know, in your blood, your body. And we see that thing sticking out of a cell — that is not [itself]. If you think about the 3D configuration of a B cell coming onto that protein, it is exactly like if it was a natural infection — which is why if you look at the data across the nine vaccines we put in the clinic, the antibody level is so high because it’s perfectly mimicking nature.

      Hanne: How did you know which protein and that one was enough? How did that process work?

      Stéphane: That’s a very good question. And, as Mr. Pasteur would say — and, of course, he has a big role in vaccinology — “only with a prepared mind.” So, one of the things we were doing with Dr. Fauci’s team for the last couple of years, is we [were] collaborating on studying viruses that could become outbreaks. None of us thought we [would] see over our lifetime a global pandemic. The last one we were all aware of as students of infectious disease is, of course, the Spanish flu. And so, one of the things where we got lucky is, we had been working for a few years together with Dr. Fauci’s team as part of that project for outbreak readiness on the MERS vaccine — the Middle East Respiratory Syndrome. 

      Which, if I had used those words a year ago, nobody would have known what I was talking about, but today everybody knows it’s another coronavirus. We wanted to provide to them mRNA for research grade — so, animal testing, antigen design, picking the protein that makes sense. Because mRNA is so easy to make once you industrialize it. We were able to send to NIH, to the team working on MERS, all the different vaccine designs they wanted to try in animals. They would vaccinate the animals and then they would challenge them by giving them a high-dose of a virus. The one that was most protective was always the spike protein. They tried a lot of combinations, but spike by itself was always the best.

      Jorge: And the theory, I assume, is because you’re essentially putting neutralizing antibodies around the spike and the spike is what the virus uses to get into cells in the first place.

      Stéphane: Correct. The full-length spike protein was always the best. Some companies went into a clinic with three, four, five candidates. And there were different hypotheses they were testing. We did not have to do that because we had tried it for a couple of years. We knew that with our mRNA, our best guess was going with the full-length spike protein.

      Success of Moderna’s vaccine in trial

      Jorge: At a very high-level, you are essentially printing these vaccines versus growing versions of a virus or a denatured virus. So you can design it, you can print it, and then you can, you know, obviously get this into people very quickly as a result. That is a remarkable part of this entire story that is probably somewhat underappreciated, that allowed you, and collectively us, to move so quickly. When did you know, Stéphane, that, all right, this is going to work, this is going to work for COVID?

      Stéphane: I had a very high belief that this should work since the beginning, so since January. Because this was the 10th vaccine we were working on. So it’s in the human data of a previous one. And in infectious disease — unlike in oncology, where the animal model tells you nothing — the infectious disease, if you look at a lot of data, there is extremely high translation from animals into humans. I saw MERS data before we started, of course, dosing in humans. So I knew the data in MERS looks great. So because we had done nine vaccines before, I knew it was going to look great in humans, which we learned all of this in May.

      Jorge: Can you describe, Stéphane, when you first saw the interim Phase 3 data and what your reaction was?

      Stéphane: So, it was a Sunday in November. I knew the independent NIH-led Safety Data Monitoring Board was going to meet at 10 a.m. on Sunday. And so I told my wife and my kids, I’m going to be a wreck the whole morning. I tried to pretend to work, but I was so distracted, I would check my email every two minutes, my phone every two minutes for a text message and so on. Maybe a bit before 1, I got a text from my team saying, “Hey, get on WebEx, we’re going to get the data.” There was not even a slide made. It was just somebody talking, literally reading to us the data.

      And so I learned about the close to 95% efficacy. It was already a big N and the p-value was very, very low. Very, very low. So this was real. And the piece that was almost the most exciting to me and my team was the severe case of disease, which there were, I think, eight or nine on the interim data. We have now 30 on the final analysis. And there were zero on the vaccine — they were all on placebo. And you think about what this means, when you connect those two data sets together, it means if you get our vaccine, you have a 95% chance of having zero symptoms if you get infected by the virus. You will not even know you are sick, you’ll just go live your normal life, zero symptoms. And in the 5% case, where you will get disease, it will be mild disease. You will get no severe disease. 

      And when you think about what has happened to our society — the elderly, people with high comorbidity, from hospitalization, when it gets bad [it] leads to death, and the total impact on the economy, the loss of jobs in so many industries, and so on — that whole cascade. If you could have a vaccine where most people, 95% get no symptoms, and the 5% who do get mild symptoms — never go and walk into a hospital — that will be a total game changer. So I listened to the data, then we talked to my team [for] a few minutes. No, I don’t think we were processing — and then I left my home office and I called my wife, she was in the house. And I told her, and I just started crying in the house.

      Hanne: I think that’s what it felt like for all of us hearing it too. It felt like, you know, normal life could return. It was the promise of something like that.

      Stéphane: We are losing, right now, 3,000 people in this country — I think it’s more than 10,000 people a day around the world. And it’s going to be a very tough winter. And that’s only the human toll, which is gigantic, but the piece I don’t think is talked about enough is the mental health toll happening to, you know, people at every age. All the young in, especially, you know, more disfavored communities where, you know, people are living in the small apartment, where Mom is trying to work. And kids trying — without a computer, without a good internet line, to learn remotely — the impact this will have in terms of equality. 

      And then, of course, so many industries have been totally destroyed. I mean, look, they are closing indoor dining again, which I think is the right thing to do. Because I think the most dangerous thing right now is to have dinner indoors. I have not walked into a restaurant indoors since March, and I won’t go until I’m vaccinated.

      Rapid manufacturing process

      Jorge: So, as amazing as I think the COVID vaccine story is, I think it’s also worth talking about the machine that made the vaccine — the technology platform that you have built over the course of 10 years that allowed you, in January of 2020, to say like, “Hey, we need to develop a COVID vaccine.” I remember coming to visit Moderna on Kendall Square, that first facility you had. And what was interesting about it is you walked in, it didn’t look like your typical biotech company. It was a row of machines, a row of printers, a row of robots. And that’s very different than what your traditional biotech company looks like. And it looked a lot more like an assembly line, in some ways. Where you can order something up and out the other end would come the mRNA medicine that you had ordered.

      Stéphane: Yes, and this goes back to this incredible property of mRNA, which I’m surprised that so many have missed — is that this is a disease and information-carrying molecule that you can industrialize. When you are in an analog business — which is what I think all pharma and all biotech is, in my book — it’s because every molecule is a different chemical entity, you cannot industrialize the making of a lot of it at the research grade. You have to literally have chemists, and pipettes, and so on. You know, doing like we all did in chemistry class, writing the synthetic route to get to a molecule that they want to do the biological effect they want. 

      And then they have to design that chemical equation, and then all the pipettes and test tubes to do that. And when it’s another molecule, they have to invent another synthetic route. So it’s very — an analog world where you invent everything once at a time for one product. Because if every product is different, you have to re-optimize every time, and sometimes it’s very complicated because of very complex biological systems. So sometimes it will take you 6 months, 12 months, 18 months to get ready from preclinical data to be making clinical grade product that you need to file to FDA so that they give you the green light to go into testing this in humans. It’s highly regulated — as it should be processed to protect people’s safety. But in our case, it’s always the same thing, because mRNA is always made of [the] four same letters — the four letters of life, like zeros and ones in software. It’s the same manufacturing process. 

      This is like software or LEGO, this is an engineering problem. It’s an engineering technology, it’s a platform. The only difference between all Zika vaccines, or all CMV vaccines, and the COVID vaccine — it’s only the order of the letter; the zeroes and ones  of life. The manufacturing process is the same, the equipment is the same, with the same operators. It’s the same thing. And so this is why we could go so fast. It took us 60 days to go from a sequence of a virus presented by the Chinese to dosing a human. The first SARS — SARS-CoV-, or as it was known before SARS — it took the NIH 20 months <Mmhmm.> to go from sequence to starting the Phase 1 study. So, you went from 20 months to 2 months.

      Possible future uses for mRNA drugs

      Jorge: Which is remarkable. Are we in the plug-and-play future for vaccines?

      Stéphane: Oh, 100%. We’re going after making a seasonal flu vaccine — because, as we all know, still 10,000 Americans die every year, on average, of seasonal flu. We believe that we should be able to make a big dent [on] flu. And today we have six vaccines in development. We’re going to have many more soon, because for 10 years, you know, Jorge, we hoped that mRNA vaccines were going to work. We believed scientifically they were going to work — but until you have a Phase 3, randomized, placebo-controlled study where you test for the prevention of disease, you don’t know. Now we know.

      Hanne: Are there limits right now to how sophisticated these instructions can get, or can we essentially give them as sophisticated instructions as the human body is capable of?

      Stéphane: It’s — when the mechanism of a disease is not well understood. So we spoke about vaccines, and we said, “Look, coronavirus,” as I said, “is actually a simple virus.” We, as a society, got lucky. Think about HIV. HIV [was] discovered 40 years ago. There is still to this day no approved vaccine against HIV. Think about the awful world we would be in right now if Dr. Fauci had been standing on the presidential podium back in spring, and told them, “Folks, I’m sorry to tell you, but this is an awfully complex virus. We have no idea when we might have a vaccine.” Think about the state of mind we would all be in now. The biology of viral genetic disease is very well understood. Why? Because kids got two biogenetic information from their parents that they cannot make a correct protein, and that is what causes their disease. They have a wrong instruction in their DNA. 

      You can give them an mRNA from our technology, coming in their cells with the right instructions — then they will have the right protein and they won’t get sick. If you think about cancer, on the other hand of the spectrum, or Alzheimer’s now, if the disease mechanism is not understood, we cannot drug it easily. We can try things, of course. We could make an mRNA behind that hypothesis, go try it in a clinic — but a lot of things will fail because we are guessing. And so the piece where I think we have an incredible tailwind — basically overlaps doing academic biology work around the world are helping us. Because if tomorrow there is a paper published by our lab in the U.S., or in China, or anywhere in the world that says protein X, Y, Z is the root cause of that disease, or those five proteins in this ratio are the root cause of that disease, then we can literally turn on the computer and, you know, design a drug to go test that hypothesis in an animal.

      Jorge: Basically, the power of this approach works when you know what you want to make and then you just need to deliver the instructions to make that. Where it doesn’t work as well is when you’re not quite sure what it is that you need to make.

      Stéphane: This is basically biology complexity or biology risk. The other dimension for us is the ability to deliver the mRNA in the right cell. We actually have become a “delivery of nucleic acid” company. We realized that what would allow us to maximize the impact we could have on disease, or helping as many people as we can over the next 5, 10, 20 years, is the ability to bring up mRNA to different cell types. So a good example today is, if you say, look, there is this university that published the mechanism of Alzheimer’s disease. If it happens in the brain and we don’t know how to bring mRNA [to] the brain safely, we cannot drug it. So the biology will be understood, but the delivery technology will not be there.

      An example where we’re making a lot of progress right now is the lung. <Mmhmm.> We have been working with Vertex around how to deliver mRNA via an aerosol via your mouth into your lung, because they know the biology very well. And we work together to develop a delivery system to bring mRNA safely into your lungs, and to bring enough mRNA at a safe dose to get the biological effect. And we’re getting very close now. Once we can prove in the clinic that that delivery system works, then the next morning you can make any other drug you want that you need to get into the lung, because it’s getting another set of zeros and ones coded differently, with the same delivery system into the lung. And that’s the power of the technology — which is why with vaccines we’re able to go so fast.

      Jorge: Yeah, the instructions have gotten so sophisticated over time that now the next sort of horizon is, you’ve got to get the vehicle for delivery equally sophisticated.

      Stéphane: We’re adding vertical, after vertical, after vertical — then we bring mRNA into a new cell type. So the vaccine is one vertical. Getting mRNA into a tumor is another vertical. We have a very cool drug, where we inject mRNA in people’s hearts after a heart attack — and here we code for a protein called VEGF, for the biology geeks on the podcast, V-E-G-F. That is a protein that we all have the instruction in our DNA, which basically tells your body to make a new blood vessel.

      Hanne: Amazing.

      Stéphane: You use that protein every time you cut yourself.

      Using robotics to manufacture drugs

      Hanne: Stéphane, you’ve mentioned, you know, kind of the fast design of the vaccine, and then you mentioned even robots printing medicines. Can we get your version of what that machine assembly line looks like?

      Stéphane: So, the robotics farm we have in our factory is basically just an assembly of robots that get instruction coming directly from computers. There’s no human interaction. And basically, you start from a piece of DNA. That is basically your template. You put that in a reactor with water. There is no cell — it’s a cell-free manufacturing process, which is why it’s so fast. And you put enzymes. And basically, what the enzymes do, they attach to the DNA, and they read the DNA template. And they quickly tell pieces of nucleic acid — i.e, the zeros and ones, the four letters of life — they bind them next to each other to make an mRNA molecule. Then the robot goes to the next step, which is you add a cap. 

      Think about it like the nose of a molecule, that you add again with another enzyme. Then what you do, you purify the mRNA. So basically, you pick the mRNA from all that water, enzyme, and nucleotide, nucleic acid, and so on. And then when you have a pure mRNA molecule, after purification, you mix it with a lipid, i.e, fat. And that fat basically goes around and packages, like in a little bowl, the mRNA to protect the mRNA in your blood, and to get the mRNA inside your cells. When it’s inside your cells, the lipid — the fat — falls apart, the mRNA is released inside the cell, and the little ribosome — the little 3D printer of your cell — is going to read that message, make the protein on demand, and here you go — the patient, the human is making his or her own medicine.

      Jorge: I remember from the earliest days you were obsessed with the operations. You were obsessed with turnaround time, with throughput, with, you know, cost per output. And the benefit of that approach is that it obviously just compounded over time. The benefit of the technology, as you’re describing it, is that you have a machine that prints the instructions that go into the cell — that uses the cell’s machine to make the medicine, or to make the vaccine. And that’s this incredibly powerful paradigm, you know, to taking therapeutics or vaccines from being very bespoke efforts to being truly industrialized, designed efforts.

      Stéphane: That’s what is really so powerful is that the whole drug process is all about information. The piece that is remarkable is you have this very modular technology, because what happens in our cells is actually extremely logical. We start from the sequence information of a virus, like in the case of a COVID vaccine, or we use the human genome. We put [it] into a technology genetic-based cassette, and then you click “order” on the computer and you go again. And that’s the vision I always had since day one. And a lot even of my scientists thought I was crazy, because this industrialized, engineer-driven approach to drug discovery has never happened [before].

      Hanne: So, Stéphane, you’ve described this process which is, you know, much more efficient, industrialized in nature, incredibly fast compared to the old process. Is there a world in which that gets even faster? Are there other things — you know, other increases in technology that would speed this up even more?

      Stéphane: Yes, so it took us 42 days to go from sequence to shipping the human grade vials to Dr. Fauci’s team. The big bottleneck is sterility testing — a very important quality control test that is done for any injectable pharmaceutical to make sure that there’s no bacteria in the product. That test takes two weeks, because what basically you do, you take a sample of your vaccine and you wait enough time. If there’s only one copy of a bacteria, by that time you have enough multiplication of bacteria, through the detection of the assay of a test that you will see it, you will not miss it. It’s very important for people’s safety. Well, if there was a technology developed where you could do sterility testing in one day with high sensitivity, then you could take our process down to two weeks.

      The history of Moderna and its approach

      Jorge: So, we’ve talked about the vaccine. We’ve talked about the machine that made the vaccine. I’d love to take a second to talk about the company that built the machine. So from the moment that you started this company, you took a very different approach. And you’ve described it as having an engineer’s mindset. Can you talk a little bit about what you did, and how you thought about the early company build?

      Stéphane: I had never built a company in hypergrowth. You know, I worked at Eli Lilly, I ran bioMérieux, which is a big diagnostic company. But I have never built myself a company building very, very quickly. We decided to do something very atypical, because most biotech companies are one-drug company at a time. What was very clear to us, because mRNA is an information molecule, is it made no scientific sense that this will be a one-drug company. It will be either zero, because we run out of money before we can safely get the drug approved, or it’ll be a company with thousands and thousands and thousands of drugs because of the platform.

      And so, once we realized that, in the first hours of talking about Moderna, we started to become very worried and paranoid about, “Geez, we don’t know what we don’t know about this technology because it’s new. It has never been approved.” And, “Geez, if we pick one drug, if we are wrong and it doesn’t work in the clinic, everybody will believe what people have believed for 50-plus years” — which is, mRNA will never be a drug. And we most probably are going to go bankrupt. But if mRNA could’ve worked, we will have failed society. Because if we find a way to make this work, this will [mean] thousands and thousands of drugs that are undoable using existing technology — like the VEGF in hearts — and we will shortchange societies, shortchange patients. And that was just unbearable.

      And so we spent a lot of time thinking about, okay, what are all the things that could make us fail? We ended up zooming [in] on four risks that we say — if we can manage and reduce those risks, we will have [the]  best chance to be the best version of Moderna. Those risks — we’ve talked about very publicly, especially when we went public. It’s technology risk around the mRNA technology. So, of course, if you do a new technology you don’t know what you don’t know. There’s going to be a lot of risk there of things not working as you expect. Two is the biology risk. You can have incredible risk that your scientific hypothesis on the biology is incorrect and the drug will fail — not because the technology wasn’t working, but because the scientific hypothesis on the biology is incorrect. Then there was going to be a lot of execution risk. And then, of course, financing risk. Because we said, like, you know, asset managers build a portfolio — we said “Picking one drug is crazy, it’s like buying only one stock.” And so we said, “Let’s build a full portfolio of drugs.”

      And after many, many months of discussion, we designed basically a pipeline of 20 drugs that we said we’re going to take all those drugs in parallel to the clinic, so there would not be a binary event that the company makes it or not on one drug. So we diversified the technology risk around six different technology applications, from vaccines, to [a] drug in the heart, to a drug in the liver for a genetic disease. And then for every application, we took several drugs to diversify the biology risk. And we launched that crazy experiment with, you know, 17 drugs in the clinic so far — which was going to create incredible execution risk because it’s harder to do 17 at the same time than 1. And incredible financing risk because we needed a lot of capital. But we traded those risks with our eyes wide open, because the other risk could kill us with much higher probability — the technology and the biology risk.

      Jorge: It’s very difficult in this industry to take that balance, platform versus programs. And, you know, what tends to be the case very quickly is most companies when they have to choose where to put an incremental dollar, or an incremental head, they put it on the programs because those are the golden eggs and they want to move those forwards to create value inflection. And as a result, the platform ends up getting starved. <Yes.> You started the other way around. You actually fed the platform, and you fed the goose, and then let the goose lay its eggs.

      Stéphane: Yeah, exactly. The goose is more valuable than any egg. If you really believe you have a goose that’s going to be making thousands and thousands and thousands of eggs, you don’t want to kill the goose on the first or second egg.

      Jorge: Although most geese are not that fertile in biotech. Yours… <crosstalk, laughter>

      Stéphane: And that’s why I told you both that I was not interested [in going] public early because the capital markets were going to force me to not invest in the goose. Because biotech firms like to bet on eggs, not on [the] goose, because there has not been a lot of geese before in this industry. So we’re not used to it.

      Jorge: I mean the record will show that you did a lot of things right. As you built the company over the last 10 years, can you talk a little bit about the things you did wrong, that if you could get them back you would do it over?

      Stéphane: Well, [the] easiest one, given the COVID situation is — it took us three years to start working on vaccines. So think about how the world would be different and Moderna would be different if we started working on vaccines from day one. We might have been able to go even faster for COVID. So that’s a thing I regret, and that’s on me. I made quite a lot of mistakes hiring people, because I underestimated how intense our company is because I live it every day. I thought initially that it was obvious that this is a small company fighting for its life, so people are going to work hard. It’s brand-new, cutting-edge science, so it’s going to be complicated because every other thing is not going to work. 

      So, being able to manage uncertainty — people having a lot of grit. Collaboration, because making a drug is a team sport. A drug is a system of so many capabilities — the biologists, the [toxicology] people, the chemists, the engineers to make the drug. And a lot of times, people coming from big pharma are used to working in silos. And people who come from academia don’t know how to develop drugs. It’s a system. And like any system, you get the best outcome if you really optimize the system working together.

      Jorge: So, last question I would ask you — what advice would you give to the engineer that wants to get into biotech?

      Stéphane: So, first he needs to learn a bit about biology. I mean, I had a chance, as I spent my entire career in biology, so I’ve learned a lot on the go — I’ve learned a lot by reading. I’m a curious guy, so I read a lot. You can get biology books and learn. And I think it’s understanding enough of biology so that you can be part of a conversation, so that you can have an impact on decisions and scientific choices that happen. And then you can go from there.

      Hanne: That’s wonderful. Thank you so much for joining us on “Bio Eats World,” Stéphane. We’re so grateful for your time.

      Thanks so much for joining us on “Bio Eats World.” If you’d like to hear more about all the ways biology is technology, please go subscribe to the a16z bio newsletter at a16z.com/newsletter, and of course, subscribe to “Bio Eats World” anywhere you listen to podcasts.

      • Stephane Bancel

      • Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

      • Hanne Winarsky

      The Cost Disease in Healthcare

      Marc Andreessen and Vijay Pande

      How come things like healthcare, education, and housing get more and more expensive, but things like socks, shoes, and electronics all get cheaper and cheaper? In this episode of Bio Eats World, a16z founder and internet pioneer Marc Andreesen, and general partner Vijay Pande, discuss the lesser known law of economics that explains why healthcare, education and housing is so damn expensive, and getting worse.

      What’s really at heart is tech’s ability to transform (expensive) services into (affordable) goods: think of the cost of a live string quartet, versus a streamed recorded track; or the cost of a custom-made shoe, versus a factory-made one. Until now, using tech to similarly transform services into goods in healthcare has seemed like an impossible dream — how would you do this for, say, the service of doctors providing care? But in this wide ranging conversation all about technology and society across all industries, Andreessen and Pande talk about the massive new gains recent technologies have begun to make this seem within reach, from eye surgery in malls to using AI in processing medical claims. Is there a future in which what doctors are doing today feels analogous to farmers hand plowing fields 300 years ago? And what would the role of that doctor of the future be?

      Show Notes

      • How Baumol’s cost disease is distorting pricing in healthcare and education [1:56]
      • How technology could reduce healthcare costs [5:52], just as it has in the past with other goods and services [10:41], and LASIK as an example of market-driven healthcare [13:33]
      • The role of individual behavior in chronic health conditions [17:20], and ideas for how this can be managed [19:56]
      • Using apps and wearable devices to drive positive behavior change [25:15]

      Transcript

      Lauren: Hi, I’m Lauren.

      Hanne: And I’m Hanne. And this is our show, “Bio Eats World,” where we talk about all the ways our ability to engineer biology and re-engineer healthcare are transforming our future.

      Lauren: So, Hanne, this episode is called “The Cost Disease in Healthcare.” What disease are we talking about?

      Hanne: It’s actually a reference to what’s called Baumol’s cost disease, or the Baumol effect, which is a phenomenon first described by an economist named William Baumol in the 1960s. In short, the Baumol effect is when there’s a rise in wages and jobs and industries that then haven’t had the same rise in productivity.

      Lauren: Okay. But what does that really mean, and what does it have to do with healthcare?

      Hanne: That’s exactly what this episode is about. a16z founder and internet pioneer Marc Andreessen and General Partner Vijay Pande discuss the economic forces that make some things like healthcare, education, and housing get more and more expensive but things like socks, shoes, and electronics all get cheaper and cheaper.

      In this wide-ranging conversation about how society and different industries work and what that means for consumers, Marc and Vijay talk about tech’s ability to transform expensive services into affordable goods. Think of the cost of a live string quartet versus a streamed recorded track, or the cost of a custom-made shoe versus a factory-made one. But until now, using tech to similarly transform services into goods in healthcare has seemed like an impossible dream.

      How would you do this for, say, the service of doctors providing care? Marc and Vijay talk about the massive gains in new recent technologies that have begun to make this seem within reach, from laser eye surgery in storefronts and malls to using AI in processing medical claims.

      Is there a future in which what doctors are doing today feels analogous to farmers hand plowing fields 300 years ago, and what would the role of that doctor of the future look like? Take it away, Marc and Vijay.

      Baulmol’s cost disease

      Vijay: So maybe, you know, the place to kick this off would be to talk about what is Baumol’s cost disease and why it’s so important. I think maybe the Twitter version of it is, how come things like healthcare and education and construction exponentially increase in cost, whereas socks from Walmart or many other goods, especially anything electronic, decreases exponentially. How could that be? I mean, we’re living in this world where things magically get cheaper, but that college education or healthcare just is becoming this massive challenge for us as a nation.

      Marc: Yeah. So, the way that you measure the impact of technological change in society is through what economists refer to as productivity growth. It’s the process of figuring out how to make more output with less input, right. And so, normally, we kind of expect the world to work this way, which is, every year over the last 300 years with a couple of exceptions, most industries got a little bit better at making things and costing a little bit less, and that led to this huge rise in living standards.

      Agriculture is kind of the classic case where food is really cheap as a consequence of a lot of technological leverage applied to the challenge of growing food, and we just generate a lot more calories of food for a lot less money now than we did 100 years ago or 300 years ago.

      The problem or the challenge is that different sectors of the economy have different rates of productivity growth, basically, depending on their idiosyncrasies and then depending on the extent to which technology is empowered or allowed to actually have its effect on things.

      And so you see these industries like consumer electronics and media and food and clothing in which you’ve got this spectacular productivity growth and then correspondingly these spectacular price declines over time. And then you have these other economic sectors — education and healthcare and housing as three in which the price curves are going in the wrong direction.

      The cost of a college degree gets more expensive every year. The cost of heart surgery gets more expensive every year, which is going backward, basically neutral, and maybe even negative productivity growth. Like, they might be getting worse over time.

      Then you basically got this problem where you’ve got certain industries that are racing ahead in productivity growth, and so, those workers are kind of becoming super technologically empowered to produce a lot more with less input. And so, those workers are actually getting paid a lot more because they just got so much technological leverage to the work that they do.

      Think about, for example, the producer of a TV show or something like the level of kind of power that you have with a modern computer to like produce a TV show. It’s leaps and bounds beyond what you would’ve had if you were literally cutting, you know, splicing film, you know, by hand with the scissors and tape, which is how things used to work.

      And so you’ve got these industries in which productivity is growing very fast, prices are declining, and wages are exploding. And then you have these other industries like education and housing and healthcare in which that’s not happening, but the problem is workers can actually migrate from sector to sector. If I’m not excited enough about having a job as, like, I don’t know, a film editor or something, like, I can go to nursing school and I can become a nurse. And now I’ve migrated out of the media industry, and I’ve migrated into the nursing sector.

      And then the problem is wages get set across these industries, and so you basically have industries with neutral and negative productivity growth that are now setting wages as if they had positive productivity growth, which they don’t. And then the result of that is just this explosion of price in those kinds of negative productivity sectors. It’s just horrible for consumers of healthcare, education, and housing because the same stuff gets basically more expensive every year without getting any better and maybe getting worse.

      Oh, and then, the other big problem is, there’s no reason why this would ever stop. The way I would crystalize this whole thing is — because of rapid productivity growth in consumer electronics, a 100-inch big-screen TV that goes on your wall, in your house, and you can watch every movie ever made for $10 a month. The price of that TV is going to drop to, like, $100. That is, like, quite literally what’s happening.

      Correspondingly, the price of a high-end private four-year university degree has leaped up dramatically over the last 20 years. It’s now in the order of, you know, $75,000 a year. So it’s like $300,000 for a degree. It’s not going to be that long before a four-year college degree costs a million dollars.

      And so you’re going to have a $100 television set that covers your whole wall, and you’re going to spend a million dollars getting a college degree. And that’s just crazy. It’s just, like, such a horribly bad outcome, and yet there’s something in the structure of how these markets work that prevents us from kind of speaking openly about the trends that result in this.

      Technology and AI in healthcare

      Vijay: Yeah. I think, Marc, one key point that you laid out there was that this is very much the cost of labor and that there’s a sort of specialized labor. And in many of these industries that we’re talking about — healthcare, education — this is an apprenticeship, where you have to spend many years to be able to develop skills that are handed down from one person to another. Very, kind of, pre-Industrial Revolution kind of behavior. Whereas when you talk about goods, goods are made in factories that are completely automated, and that technology can be applied there to make them 10% better a year, and that leads to exponential performance over time.

      And one of the key ideas that I was curious to hash out with you is what we’re seeing in AI. What we’re starting to see is that AI is turning what used to be something that had to be done by a service into something that can be thought of as a good. That instead of a person training in an apprenticeship-like way to do something, the machine learns these things. You can make copies trivially. You can get now the advantage of Moore’s Law, and this almost alchemical magical transformation seems to be one part of a potential solution to addressing Baumol’s cost disease. I’m curious how you see, at least, that part of it?

      Marc: Yeah. So this actually goes to an example that Baumol used when he wrote the book, kind of on this topic. And so he used the hypothetical example of, like, a string quartet, right. There’s two ways to experience a string quartet, right, in your house. One way is the old way, which you can actually hire four musicians to come and set up in your house and play Beethoven quartets, and it’s going to sound great. And by the way, people still do this. You do this for, like, weddings, right, this is still a thing.

      The other way to experience a string quartet in your house is electronic playback, a recording. And what’s the cost of a string quartet recording to playback in your house today? You know, it’s basically zero. If you just chart the price of getting four musicians to come play at your house from, like, 200 years ago to 100 years ago, to 50 years ago, to 20 years ago, to now, that price has exploded. The in-person version has gotten, like, wildly more expensive, right, because of Baumol’s cost disease, because those musicians actually work for a living, and they have other career options.

      And then, exactly to your point, in AI, what’s the simplest form of AI? It’s a computer literally listening to what’s happening and playing it back. And it turns out that costs nothing. An enormous amount of progress in the modern economy is that, right. You also benefit from that, by the way, every time you buy a loaf of bread. Our ancestors were not buying loaves of bread carefully pre-sliced off the shelf. Our ancestors, to the extent that they were able to get access to the core ingredients in the first place, were, you know, making bread by hand.

      Vijay: Yeah. Baker as a service.

      Marc: Yeah. Exactly. This is actually the big lever that increases living standards. Exactly to your point also, it has been hamstrung by the fact that historically computers have only been able to do so much. Machines have only been able to do so much. And now we have these sort of much more flexible technologies kind of gathered under this term of AI that at least in theory give us the opportunity to now revisit a lot of our assumptions about what should be a product and what should be a service.

      Vijay: Yeah. One of the fun things that we’re seeing is AI is nibbling in with the easy, mundane things that are annoying for people to do that they have to be trained to do, but then, that training that goes to people can now be done to the AI, and the AI can be trained once and then scale and actually learn from everyone else’s mistakes.

      And so what we’re seeing, as initial go-to-markets in healthcare are in areas of billing or simple types of diagnoses or triage — areas where this isn’t trying to make some superhuman genius, which may in time come, but I think the first go-to-market is just taking the things that are just boring and reproducible that are just expensive because of the human power involved, not even necessarily because you need a super genius. And that’s something that we’re seeing right now.

      Marc: There’s a famous story of Alan Turing where he was working on inventing the computer in the early 40s, and he was hanging out at Bell Labs in New Jersey with his friend Claude Shannon who’s the inventor of information theory. The two of them were having this heated lunch discussion at the AT&T headquarters building in New Jersey with all the top researchers and AT&T executives kind of sitting around nearby about basically this concept of AI. Like, what would it mean for computers to actually be intelligent to actually have brains.

      And they were debating back and forth, and finally, Turing got frustrated, and he stood up and yelled at Claude Shannon. He said, “Look. I’m not talking about turning a computer into a super genius. I’m just talking about building a mediocre mind, like the president of AT&T.” And this gets into the emotions and the politics of how we think about automation, because the technological progress and productivity growth changes jobs, but in the fullness of time, what we will realize is that a lot of what doctors are doing today, for example — a lot of that work is going to be analogous to literally when farmers used to hand plow fields 300 years ago.

      Like, if you took a modern farmer who’s running a fully computerized operation with all these modern combines and tractors and GPS and all these amazing hybrid engineered seeds and all these miracle fertilizers and everything, and if you told them that they had to go back to hand plowing fields, we would have much worse food at far, far higher prices and a lot of people would go hungry.

      I am quite convinced the doctors in, you know, whatever, 50, 100, 200 years are going to look back at what doctors do today, and they’re just going to be, “My God. I can’t believe those poor people ever had to do all that.” And in fact, they’re also going to say like there was so much more important work to do.

      Vijay: Yep. You know, it’s interesting to think about what the arguments against this could be, and one would be that — well, you know, could the industrialized process be really comparable to what a human being can do? People can do so many things. I was just thinking about how shoes were made. You would have a cobbler who could make shoes that were perfectly suited for your feet, and they’d be doing — everything would be one-off and bespoke and probably would be better shoes, maybe. But instead, you just define a series of shoe sizes — you know, I’m either like a 9, 9.5 or a 10 or whatever, and I could just get the closest one, and it’s good enough.

      And the fact that it’s 10 times cheaper or whatever, and now with non-material so much better, that pretty soon you forget about the other experience, and you just get used to the new way of doing things. And that’s kind of my suspicion, that in the beginning there will be trade-offs that you have to make, and that people will have to get used to, but that in time I think you wouldn’t think of doing it any other way. And at least this follows industrialization in other contexts.

      Marc: Yeah. And in fact, back “in the glory days” when like all shoes were made by hand, they were sold, like, [so] crazily expensive that you would have one pair of shoes, right. This idea that you’d have like a shoe closet would’ve struck people as just absurd because you have a pair of shoes. And then, by the way, they’re so expensive because of all the manual labor involved, right, relative to your ability to make money, you know, as sort of a normal worker that like if your shoes start to wear through the sole, you’re out of luck. You’re probably going to be wearing those things for five years.

      Kids wearing, like, newspapers stuffed in their shoes, right, to be able to basically compensate for the holes in the shoes because shoes are just a lot more expensive to replace. Just imagine that shoes cost 1/6 of all GDP, right, which is where we’re at with healthcare, right. And so imagine if it was like 1/6 of all economic output had to be used to pay for shoes, and then it turns out nobody wanted to pay for anybody else’s shoes — and how terrible that world would be. And how that would really screw up, you know — we’d have all these crazily intense, like, political debates. We would’ve had these political debates between Trump and Biden on the national shoe policy.

      Vijay: Oh, yeah. Yeah. Obama shoe, Trump shoe.

      Marc: Yeah. Exactly. And then, you know, government-made shoes, right, getting these things out of the realm where you have to have these debates because things are like gigantic, expensive and nobody wants to pay for them is itself just a massive [inaudible] function increase in human welfare that you don’t notice it until you don’t have it.

      Vijay: Well, that’s why I think it’s maybe not as much of a surprise why it’s showing up in healthcare because healthcare will eventually become 100% of GDP.

      Marc: Right.

      Vijay: So it’s something that’s not sustainable, this exponential growth in costs. So I think entrepreneurs are seeing that potential. They’re creating this in both front office for doing scheduling, for doing diagnosis, for doing back-office, for billing — all the sort of routine and horrible things that people hate. But I’m curious, let’s just posit that the technology will continue to advance and that AI will get a foothold and will do something and then eventually more and eventually more. I’m curious, Marc, if you think, is that it? Is that enough where AI is doing some large fraction of the work to really shift this cost curve, or is there more than just a technology that’s required as part of the solution?

      LASIK as a case study

      Marc: We actually do have a clear example of this happening in the area of medical treatment, and that example is laser eye surgery, right. Basically, LASIK — laser eye surgery which basically literally will fix your eyes, so you don’t need glasses anymore — is the kind of medical procedure that if you described it to somebody from 1950, they’d think you’d lost your mind. It’s literally beaming lasers onto the surface of somebody’s eye to change the shape of the eye.

      Vijay: In your mall.

      Marc: Yeah. In a mall, right, quite possibly right there in the front window. Right?

      Vijay: Yeah.

      Marc: And so, it’s an amazingly technologically advanced procedure. It’s actually gotten even more technologically advanced over the last 20 or 25 years. There was a point when you had to, like, try to hold really still because the laser needs to hit the right part of the eye. And now, they’ve got all this advanced 3-D cartographic mapping where the laser follows your eye movement in real-time. And so it’s become this incredibly technically sophisticated kind of thing. And while that’s been happening, the price has been dropping.

      And in fact, the reason why LASIK outlets are in the mall is because they can afford to be, right? It’s actually become quite inexpensive to set up a LASIK operation, and it’s actually quite inexpensive to get LASIK. This is the kind of thing where it’s like, this procedure as a technological feat is not more advanced than heart surgery. It’s not more advanced than certain forms of even, I would speculate, brain surgery. This is advanced stuff, and yet this thing is on a quality improvement and cost reduction curve completely unlike any other surgical procedure.

      And then you kind of say, well why is that? And, of course, the reason is because it’s paid out-of-pocket, right. So it doesn’t run through the insurance system. It’s not something that other people pay for. It’s not something that has any politics around it. It’s an outpatient procedure. It’s voluntary. And if you don’t get it, by the way, then you get glasses. And if you do get it, then you don’t need glasses.

      And so, as a consumer, you can actually make the trade-off of, like, is it worth to spend whatever — $1,500 for this surgery, as opposed to spending, you know, $200 for glasses every few years. What if we could basically re-engineer our whole approach to how we think about all this stuff? And, you know, we can’t literally do that, because consumers might be in a position to decide whether they need eye surgery and how that should work. Maybe they’re not quite in the same position to understand what it means to have a quadruple bypass. And then there’s also, like, it’s an outpatient procedure. Inpatient procedures are a lot more expensive, have, you know, lots of care requirements. But nevertheless, it’s like, “Okay. There’s a shining beacon for what’s possible.” So there’s that.

      There’s also this big definitional question in my mind which is, like, what is healthcare? And we tend to think that healthcare is like a discrete thing and the politics are kind of all calibrated around that. And so the big political arguments are always about, do you have healthcare, or do you not have healthcare — as if you’re saying, like, I don’t know, do you have a shirt or do you not have a shirt, right? But that’s not really what it is. The definition of what it means to have healthcare keeps expanding, right, as sort of the number of things that people consider to be conditions that they want treated and the number of things that are actually treatable keep expanding.

      And then there’s this whole other debate of inputs versus outputs, which is, how are we measuring healthcare? Are we measuring it by how much it costs and all of the things that go into it and all the procedures, or are we measuring it in terms of outcomes and literally things like health and longevity, right, and sort of physical vitality? And you really start to have different views on basically what it is we’re all paying for, what value we’re getting for it, and then, by extension, what shape and form healthcare will have in the future where it could end up being very radically different.

      I’ll just give a thumbnail sketch for how healthcare can be radically different in the future. It may be that all the medical procedures, surgical procedures get basically automated and become very cheap, but it may be that we end up spending more healthcare than ever because healthcare basically turns into advanced therapy. And so instead, like I said, it may turn out to be the physical issues are the easy and cheap ones to deal with, and it may turn out that it’s the psychological and sociological issues are the complex and painful ones to deal with.

      And so, maybe the job in the future of “doctor” and “therapist” merges, and we end up with this very different type of profession that’s really oriented around helping people optimize their entire life. And then it’s like, “Oh yeah. Every now and then, you need to go get a little laser surgery, but that’s, like, not the major part of the spending.” And then as a consequence, like, maybe doctors, you know, 20 or 50 or 100 years from now are paid a lot more, because they’re actually a lot more valued in our lives despite the fact that so much of the work that they do today has been automated.

      The sheepskin effect

      Vijay: Yeah. That they become the focal point for all that automation and keeping the human element. And your point about inputs and outputs, I think, is super important, because if you compare it to other areas where Baumol’s cost disease exists, like education — that also seems to be very much measured more by inputs and outputs. People ask, “Oh, did this school get as much money per student than that school?” Not, “How well did the students do?”

      Marc: So, the crack in the matrix that makes me really wonder about education as a service, as a product, or whatever, is something called the sheepskin effect. And so, basically you assume that, you know, you go to school for eight semesters, you know, four years. You come out the other end, you get a job, and let’s say the job pays you whatever — you know, $80,000. So then, apply the following thought experiment, which is, what happens to that income once you’re out of college? What happens to your rate of income if you only complete seven out of the eight semesters?

      Logically you would think, “Well, if the value of the education is all the stuff that they’re teaching me, then I’m going to get 7/8ths of the wage, and I’m going to be making $70,000 instead of $80,000,” right, or whatever that correction is. Of course, that’s not what happens. What actually happens is if you only complete seven semesters out of eight, you’re going to get paid $40,000, right, because you’re going to be a college dropout instead of a college graduate.

      Vijay: And you get paid basically what you would’ve if you didn’t go to college.

      Marc: If you didn’t go to college. Exactly right. And so that’s the sheepskin effect. There’s two possible explanations for the sheepskin effect. One is, all the actual skills are taught to you in that last semester. That’s one possibility, but we know that’s not true. And so the other explanation is, college actually does not have that much to do with the skills that are being taught. It’s something else. It’s basically a stamp of approval that says you can execute a task all the way to completion. The education may be somewhat beside the point. It may just simply be the fact you demonstrated you can get through a program.

      The healthcare crack in the matrix to me is the fact that it used to be the medical conditions that mattered were things that just happened to us that we had no control over — or you’d be in a factory, and your arm would get cut off, or you would just die, and you had no control over it. So many of the medical conditions that we’re dealing with today as individuals and societally are as a consequence or downstream in behavior.

      Vijay: Yes.

      Marc: Obesity is the big one, right. It’s like obesity is cross-linked to all these issues, right, including heart disease and stroke and cancers and, like, everything.

      Vijay: Massive comorbidities all over the place.

      Marc: All over the place. And so the most effective form of healthcare is don’t eat bad foods and then exercise every day, right. And then, if you’re going to drink like only drink a little bit, and by the way, don’t smoke. “The healthcare system,” as we understand it, is that by the time you show up having had a heart attack or whatever, you had 30 years of basically bad behavioral characteristics leading up to that.

      What does it say about us that we treat the healthcare system as basically the last-ditch attempt to keep us from dying after we’ve basically spent our life behaving very poorly. And that goes back to this idea of the doctor becoming the therapist. The answer to the actual health outcomes is upstream of what’s happening in the healthcare system.

      Drivers of chronic health conditions

      Vijay: And it goes to the bigger issue which you asked about, which was, what is healthcare? Because there’s this kind of mind-blowing article that came out that talked about the reasons for death, and what healthcare deals with is relatively a small sliver of the pie compared to genetics and social determinants. And social determinants being the biggest pie piece, 40%. If your spouse smokes, you’re probably going to smoke, or you’re going to get a lot of secondhand smoke. If your spouse is overweight, you’ll probably be overweight. If your friends drink heavily, you’re going to drink heavily — that the social determinants around you have a bigger impact on healthcare.

      And actually, we’re starting to see now when finally the healthcare companies are going full-stack. You’re seeing payer/providers thinking about an air conditioner as therapy or as a therapeutic, because that actually has a greater chance of decreasing mortality or decreasing the chance of going to the hospital if you’re living in Florida, for example, than other things.

      And so, I think that’s really kind of a key point that we need to sort of think about, and it goes to the market. And part of the challenge here is that healthcare itself is this kind of artificial market that’s created by the government where certain things are healthcare, certain things aren’t healthcare. We’re seeing Medicare Advantage and other things that allow you to go full-stack affecting this, but part of maybe now that after we get the technology in, it seems like there is no choice but to really revisit what is healthcare.

      Marc: It’s like, okay, then, how do we think about paying for this? What are we paying for, right? What are we getting for what we’re paying for, and then, of course, like, who’s paying for it? And, you know, I would just propose when you have a system that’s 1/6th of GDP in which, like, a gigantic amount of the adverse outcomes are being caused by people’s behavior or social context — and most people’s healthcare is being paid for at least in part or potentially entirely by other people, and where consumers have basically surrendered to the system and don’t feel like they have any choice whatsoever, and don’t exercise any choice. And then you have a system as a consequence that’s so heavily regulated and subsidized by the government, you can actually say it’s a minor miracle that it works as well as it does. We basically just designed the worst possible economic configuration for industry.

      Vijay: Well, and it’s funny, because some of these things are hidden almost like the germ hypotheses where people didn’t realize there were germs. That was really the hidden danger that we weren’t doing, and really sanitation is the way to fix things. It could be that eating healthy and avoiding Type II diabetes is the new sanitation. The AMA feels that they have a new initiative that nobody in America should die from Type II diabetes. And if you think about it logically, that makes total sense. Just like no one in America hopefully died from issues from sanitation the way we would maybe 200 years ago. I think maybe now it suggests that even now when we have the technology, the question is, how can we go from where we are now to this direction that we’re talking about.

      Marc: Yeah. So the positive view there was an economist named Herbert Stein who had this famous thing when he talked about these issues. He said, “If something can’t go on forever, it will stop.” And so maybe contrary to what I said earlier, if there are no limits on how far this can go, healthcare being 1/6 of GDP becoming 1/3, becoming 1/2 — at some point it becomes the most important issue in the world and people are just not going to be willing to put up with it. Maybe just the pain — like, the economic and political pain just gets, like, simply too intense and then you start to realize you have to kind of unwind yourself from some of these assumptions.

      But I think honestly the other thing is just more things like LASIK. More things where we can actually demonstrate what happens when technology kind of hits in the positive way. Like, what technology does, right — dramatic boost to productivity growth — which means dramatic improvements of quality combined with, you know, dramatic decreases in costs. The optimistic view there would just be, like, as consumers we’re getting trained to basically be able to like comparison shop and evaluate and get, like, ratings or reviews on everything in our life. Literally everything, almost anything you choose to do, whatever restaurant, you know, you go on Yelp or even these days online dating. You’re used to a level of kind of consumer choice and selection and quality control and decision-making.

      If you go, like, buy a new car or something is just, like, the wealth of data that’s available to you — it’s extraordinarily unlikely that you’re going to buy a new car these days and be disappointed, just because you’re going to know everything ahead of time and you’re going to figure out how to get the best possible deal for exactly the car that you want. I think the other part of it is supply-driven, which is just, like, we need to actually drive more technologically superior approaches into the market and like make them available and make them obvious. Like the payer/provider model you mentioned, of just — align with the people who actually want to improve outcomes and just kind of demonstrate the new way.

      It’ll be a little bit as if we had only ever had state-controlled media or something, and then all of a sudden somebody kind of had the crazy idea to like maybe actually start making movies in the private sector, and then it just turned out that those movies were 1000 times better. At some point, we just may need to make the new movies.

      Vijay: Yeah. And also, what you’re describing is full-stack healthcare on the enterprise side, which an employer will have, but also direct-to-consumer healthcare, and that we’re probably going to start to generalize that. We may start to view Peloton and Peloton-ish things as direct-to-consumer healthcare. I think part of the challenge is that — and this is true for diet and other aspects of healthcare — is that things are so tailored to the individual that it’s been so hard. And nutrition, we can do a whole podcast just on nutrition and the sort of mess that is, but I think now with all that you can measure, even to the point where you could have like a continuous glucose monitor on you, and measuring that every minute, and having that tell you what you should eat, how you should exercise.

      As we move into that, sort of, something in between LASIK and Peloton, we’ll start to emerge where maybe it’s not surgery in your eye at your house, but things that are much more clinical and that are getting to these social determinants. Getting to exercise, getting to Type II diabetes and all of its morbidities, getting to diet. You could deal with several of the top killers. That at least would be such a fundamental transition and would be the type of thing that could bend the curve that we’re talking about.

      Health apps and wearable devices

      Marc: A lot of what you just described can actually be done today. It is actually fairly amazing what you can have, like, as a consumer today just to go through the list — these fitness trackers have gotten really good, whether it’s the Apple Watch or the Fitbit or whatever. They’re now doing, you know, pulse, they’re doing blood pressure, they’re doing kind of comprehensive health state tracking, and they’re doing sleep tracking.

      So, you’ve got all the sensor platforms kind of in that thing. You’ve got the sensor platform on the phone. You can’t do laser eye surgery in the house, but people should be able to do eye exams, right, because you basically now have medical-grade visual sensors in the camera. And then you’ve got continuous glucose monitor kind of thing. And then, on your phone, you can have the fitness app that basically tells you what to do to stay fit. You can have the food app that basically helps you figure out what to eat. You can track every aspect of your behavior. You could track alcohol consumption or whatever other recreational whatever. You can aggregate all this data up, and there’s like tons and tons of apps on the phone now that will, like, basically do all this for you.

      Now, you know, the people doing this today are like the hyper conscientious types that are super into optimizing every part of their existence. That’s only a small sliver of the population that will voluntarily do this. You can just imagine a mandate, right, for people to get “healthcare coverage” or healthcare insurance at some point, you know, they have to kind of sign up for a better kind of personal behavioral regime, and they might use these technologies to support that. Or, by the way, you could imagine the voluntary version of it.

      One of the sort of consequences of healthcare being so expensive right now is this incredible rise in the individual deductible. It might be that the deductible for you just, like, laying around eating Cheetos and smoking pot is, you know, $1,500 or $4,000 or whatever, but the deductible for you with a healthy lifestyle is $200. And then you’ve got, like, the so-called good driver discount that they do for car insurance. And so then you have this sort of behavioral kind of push to be able to directly save money. And that’s an enticing idea, because that aligns the interests of the consumer, right, with the interest of the system and kind of maybe could throw things back into some kind of calibration.

      Vijay: Yeah. You think about all the parts you just talked about, that you can get this to be more consumer-driven in a market-like way. Take your previous example of the string quartet that’s in your pocket with Spotify. Now you have your doctor quartet or orchestra in your pocket — with you all the time, giving you the cure that you need. We have the motivations. We have the technology, and actually, we have the startups building it. The optimist in me sounds like this is going to happen, and this is happening, but we just have to sort of get all the pieces together to make it happen.

      Marc: So, I have forced myself to watch some cable television for the first time in a long time over this election. So, I’ve actually seen some TV commercials for the first time in, like, a year.

      Vijay: Oh, wow.

      Marc: By far, the best part of the election coverage was the meditation app Calm.com. And actually, their commercials are actually quite nice because it’s just literally, like, 30 seconds of rainforest sounds. There was also this company called Pray.com which is an app to help you pray, like, if you’re religious. It’s got, you know, Bible study and guided prayer sessions and stuff like that. At first, I was like, “Okay. That’s a weird juxtaposition.” And then I was like, “Oh, no. I get it. Calm is basically selling secular prayer, right, or Pray.com is selling religious meditation.”

      Vijay: Exactly.

      Marc: Which actually bears on health, right, because a huge driver of modern health conditions is basically stress and inflammation and, like, there are physical components to that, but there’s also a medical, psychological, sociological component to that. And so, if people are able to actually deal with stress in their lives, that could actually, like, you know, affect some of these things if it affects things like the rate of heart attacks. It can also affect things like stress-eating, which then leads to obesity.

      Vijay: Absolutely.

      Marc: It may be that the upstream apps that are, like, the key healthcare apps that we actually need on all of our phones are — take your pick — Calm.com or Pray.com. You could hire a pastor or a preacher or a priest to come to your house and pray with you or whatever advanced meditation, Zen Buddhist meditation, but it’s going to be a lot cheaper if it’s an app in your pocket.

      Vijay: Yeah. They’re just probably aren’t enough to go around.

      Marc: The serious part of this is what technology should do is it should empower us, right. It should basically give us capabilities, and it should give us reinforcement and expansion of our capabilities, and help and assistance in ways that make our lives directly better. And I think there is a very big reason for optimism that there is sort of this complete set of ways that we can actually improve our lives that the technology can really help us with.

      Vijay: Yeah. Absolutely. Amen.

      Lauren: Thanks so much for joining us on “Bio Eats World.” If you’d like to hear more about all the ways biology is technology, please go to subscribe to the a16z Bio Newsletter at a16z.com/newsletter. And, of course, subscribe to “Bio Eats World” anywhere you listen to podcasts.

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      The ‘Holy Grail’ of Social + Fintech

      Anish Acharya, D’Arcy Coolican, and Lauren Murrow

      Social Strikes Back is a series exploring the next generation of social networks and how they’re shaping the future of consumer tech. See more at a16z.com/social-strikes-back.

      The intersection of social and finance—as well as shifting attitudes around what we share about money online—have given way to an ambitious new wave of financial products.

      While revealing one’s financial information was once considered taboo, now people are more apt than ever to openly discuss money online, particularly Gen Z and millennials. That’s evident on both ends of the spectrum, whether people are bemoaning their crushing levels of student debt on Twitter and Instagram or bragging about their latest stock trades on WallStreetBets. The repercussions extend far beyond social media, fueling a wave of new social-fintech products like Public, Commonstock, and Doji, among others.

      In this conversation between fintech partner Anish Acharya, formerly a product manager at Credit Karma, consumer partner D’Arcy Coolican (who himself is a former founder in this space), and host Lauren Murrow, we discuss why the “holy grail” of social plus finance is both so challenging and, potentially, so rewarding.

      This episode was originally released last year and been resurfaced as part of Social Strikes Back, a16z’s new series exploring the many ways social networks are shaping the future of consumer tech.

      Transcript:

      Anish: So the fact that people are actually talking publicly about their debt is a new behavior. In the past, spending was public but debt was private. For the first time, debt is starting to become a public conversation. What’s new is that this generation is living in a completely different socioeconomic context. That’s not “flighty millennials” and Zoomers or whatever, that’s a completely different financial world that they’re growing up in and that’s driving a different set of conversations.

      D’Arcy: You see it across all of the platforms, but you see certain categories that people are now talking about that they didn’t talk about before. Salary is something that a certain generation is much more comfortable talking about, student debt is a category that people are much more comfortable talking about. Trading is a category that people are much more comfortable talking about.

      Across the spectrum you see sharing on social of financial stuff going up. You see it on Twitter, you see it on Facebook, you see it in blogs. There are a bunch of pockets.

      Lauren: Why do you think this shift is happening?

      D’Arcy: I think it’s driven by a few factors. One is generational, so every generation’s relationship with sharing and every generation’s relationship with money is different. So what Boomers did versus what Gen X did versus what millennials do versus what Gen Z does is different, and I think you see this macro trend around increased sharing.

      Lauren: And that’s driven by historical changes, that’s driven by the financial crisis.

      Anish: Yes, exactly. They have to take nontraditional paths to achieve financial progress and dreams. For a long, long time, buying a home was not only the American dream but something you achieved through the traditional financial system. So, everyone had a mortgage. Today, mortgages are less accessible than they’ve ever been. Will you talk to your peer set about, “How am I ever going to buy a home?” That’s really the catalyst behind many of these things.

      D’Arcy: And I think you see that also with the massive increase in student debt over the last 10, 15 years. It’s reaching unsustainable levels and that’s forcing a conversation that breaks down the stigma around talking about student debt. Once you break the stigma, then it’s like, hold on, and everything comes flooding to the forefront.

      Lauren: We’ve talked about how money is inherently private. Do you not think that that is becoming less so? There’s the generational piece of it. Then, yes, we’re sharing more of our lives in general. And then there’s a political angle to it, this idea of radical transparency to affect change. So that’s why we’re posting more about student debt, about medical debt, about our salaries.

      D’Arcy: Definitely there is a long-term trend line towards sharing more rather than sharing less. But you see it happening at the category level and, to a certain extent, at the subculture leve. Let’s take student debt as like one category. When people start talking about it, then everybody feels empowered to talk about it, right?

      I think you need catalysts for walls to come down around certain categories, like the student debt crisis, the financial crisis, there’s a lot of external events that have led to some of these things coming down. But it’s happening inch by inch and category by category. The question is: what pieces are going mainstream?

      Anish: I think the hacker mindset has pushed outside of software and into finance. There was always a small number of people who were excited about “hacking their money,” but now that’s becoming a more mainstream concept. So the idea of being someone who arbitrages rewards across credit cards used to be a pretty niche, edge thing, and now more and more people are doing it. To the point where a lot of card companies are having to pull rewards back because there’s Points Guy and a million other sites that tell you how to hack the system. And credit scores are very similar. It’s not a destiny, it’s a game—or it’s at least closer to a game than a destiny—and more people are talking about the ways that you play it. When I say it’s a game, I say that in a hopeful way, not in a dismissive way, in terms of the importance of it.

      D’Arcy: What are the things people like to do on social? Three of the core functions are bragging, complaining, and rubbernecking. And I think you’ve seen that where social and finance intersect, they’re coalescing around those three use cases as well. At the end of the day, social and finance, a lot of it is just content. It’s content that’s anchored around some financial transaction, but it’s still just content, so the usual rules of social apply.

      Another way to think about it is: when you’re building something in social plus finance you have an interaction layer and you have a transaction layer. And the interaction layer is built around the emotional and cognitive pieces—that is content creation, that is messaging, that is all these social things that we see pop up—they appeal to these cognitive and emotional levers. And then you have a transactional layer, which is whatever your actual financial transaction is. That’s generally much more of a functional use case.

      The magic in social plus finance happens when the transactional piece and the interactive piece are mutually reinforcing. That’s where the flywheel on social plus finance really starts to spin aggressively.

      Lauren: Can you give me some examples of particular products in which you’ve seen this magic happen?

      D’Arcy: The easiest example is probably Venmo back in the day. You had messaging apps and money transfer apps like PayPal that existed—and chat existed—but the idea that you could attach your transaction to an emoji just made the transaction easier, it made the emoji more fun, it made the whole thing more self-reinforcing. It’s a really challenging problem to be able to do that, but when you do it, it’s magic.

      Anish: I actually think that those products are fascinating. I still like to scroll through the global feed on Venmo, which now is capped, I think, at the last 50 transactions. But it’s just so fascinating to see all of these people all over the country sending each other money. There’s something that is just vicariously thrilling about it. And because money does touch all of us and it’s so private, the products that can start to invert that touch a nerve in an interesting way.

      By the way, it doesn’t have to only be online—there are a couple of interesting offline examples. SoFi, which is really in the business of refinancing mispriced student debt, built this whole community of HENRYs—High Earning Not Rich Yet. They did a ton of parties and events and made it feel special to be a SoFi member. And, really, they were a lender. So I think at least in the early days, they’ve had a lot of success combining the two. I imagine what’s less successful is, you know, Capital One opening coffee shops where you can hang out and get coffee and do your banking. It’s easy to dismiss that as clumsy, but I do think that they’re trying to touch the same nerve.

      D’Arcy: There’s also this long legacy of companies starting out at the nexus of social and fintech and then eventually moving one way or the other, generally towards the fintech/transactional layer. So a lot of people build either social features or community in the early days and really use it as a way to bootstrap their product, but then over time they migrate more towards a transactional fintech product, rather than a truly social product.

      Lauren: What are some of those examples?

      D’Arcy: SoFi is a great example of that. It’s functionally a lender, which is not a multiplayer social game, but they were able to build this early community which was able to get them a lot of traction. You look at like Wealthfront. Before it transitioned into Wealthfront, I think it started as KaChing, which was a social fintech product. If you look at Robinhood, originally it was a much more social product, then became a much more transactional product. Prosper started out as a much more social product, then became more of a peer-to-peer lending platform.

      So a lot of these things start social and are able to bootstrap in their early days off of some of those networks. Then you end up at a decision point where you try to thread this needle and continue down this social plus finance angle, or do you move into a more single-player fintech product? And I think a lot of the more successful fintech companies started social, but then eventually transitioned.

      Lauren: Why are they making that transition?

      D’Arcy: It’s hard.

      Lauren: Well, let’s talk about it. What’s so hard about social plus fintech?

      Anish: The most direct manifestation of social plus fintech is: we have messaging, plus we have payments or some other shared accounts, shared ledgers, joint accounts, etc. I think that is very difficult for a number of reasons. Because money is so private, people are less likely to send invites to each other and bootstrap a social product in the way that you would bootstrap other social products.

      I think there are a lot of other examples, though, where the experience may not directly represent social plus money, but it very much plays to that. So I think the example D’Arcy brought up is great, which is Robinhood. There’s been a ton of talk about how Robinhood is doomed because others have cut fees and adopted their business model. But in truth, Robinhood is a game and it’s a game that people like to talk about. It works because it feels like adulting when you actually have a stock portfolio, not because active trading is something that’s smart for almost anyone to do. So I really see it as addressing a different consumer need than Schwab is addressing, and it’s really not threatened as much by players like Schwab. So that’s an example where the fintech product is addressing a social consumer need, but at first blush, it may not appear to be the combination of social plus money.

      Lauren: And some of these products are really tapping into the trend towards gamification. Do you think more products will go that route and design around that impulse?

      D’Arcy: I think the thing you will likely see is that social plus fintech products will actually come much more from the consumer side of things. There are some things like Robinhood, where you’re able to build a fintech and community and it comes from the fintech side of things. Another encouraging angle is the things that are coming from the social sites, whether it’s a bunch of the chat apps that now have wallets and payments installed in them or even something as weird as Fortnite, which is technically a game, but they have V-Bucks and they have economies built into them. It’ll be fascinating to see what happens with those types of products, because that could be the place where we see social plus money take off.

      Anish: I do think, by the way, there have been a bunch of past attempts which maybe seemed naive at the time, but now just seem like bad timing. So Blippy is a famous example of this, where it tweeted everything that you bought. You’d link your credit card and every time you swiped it, it tweeted. Okay, like there’s obvious reasons why that might not be a good idea. And yet I think you’re like the fact that…

      Lauren: Just too soon.

      Anish: …that Dave Ramsey exists and people are talking about debt and spending, you know, there’s the nugget of truth in all of these things. And as Marc says, it’s rarely that the idea is wrong, it’s usually that the timing is.

      D’Arcy: One of the interesting things about this category of companies is that if you just take a step back and you’re looking for broader consumer trends, you can often look to little emergent behaviors that are happening somewhere on the internet and try to figure out: is that going to actually go into the mainstream at some point? One of the interesting and challenging things about like social plus fintech is that so much of it is driven by norms. So much of it is driven around what’s taboo and what’s stigmatized, and that actually exists at the subculture level.

      You can grow up in the same town at the same age, and if you grew up on one side of town, your norms around money and sharing are very different from the person on the other side of town. And so that leads to a lot of very distinct subcultures within different pockets on the internet. One of the more entertaining one is WallStreetBets on Reddit, where people are posting some mix of fake and real trades and explosions and everything like that. And so then you can look at these things and say, “Oh here’s this crazy emergent behavior that’s happening. I think this is gonna go mainstream.” In some cases it will, or in some cases it’s just part of that subculture, because the norms and taboos will never translate into the mainstream. But when those stigmas fall then, you know, everything happens and everybody runs for the entrance at that point.

      Anish: It is interesting, you know, if you think about crypto. So there’s crypto as a computing platform, which is how we talk about it a lot internally, but then there’s also the sort of socio-political, perhaps anarchist thread of crypto, and I think the historical example of that was mostly gold. You know, though at the end of…

      D’Arcy: But nobody was, like, screenshotting their Boolean collection and sharing it on Twitter.

      Anish: Well, depending on what Facebook group you were in. So I think, again, there is a past precedent. But you’re right, there’s a functional aspect of hedging against things that may go badly wrong in the future, and then there’s the cognitive-emotional and sociopolitical, to your point, Lauren.

      D’Arcy: Crypto’s fascinating because it’s a subculture that has a totally different relationship with transparency and anonymity and all of these different dimensions. Just changing the form factor of value from a dollar to some sort of token has freed an entire segment of people to talk about it and have a different relationship with it. It’s one of the most entertaining parts of social, what’s happening in crypto. And again, the concept of crypto versus the concept of money created a psychological shift in some people that then made the norms around it much different.

      Lauren: So you’re saying there are these subgroups, little niche categories, but it’s difficult to build a business around them until they reach that tipping point.

      D’Arcy: I actually think you can build great businesses around some of these subcultures. There’s a lot of this “niche,” but they can be massive niches, right? Like, WallStreetBets has something like 800,000 members.

      Anish: People always want to talk about how they’re making money. It’s having debt that’s always been private. So the hardest problem in terms of social and money is having people talk about their debt, which is why people don’t want to have a relationship with their lender or talk in too much detail about their credit card debt. They feel bad about it, they feel like it reflects poorly on them. I was just checking Instagram right now, and there’s 675,000 posts for #debtfreejourney. This has become a public conversation, and a lot of it is happening on Instagram. I think that’s the hardest problem, the hardest segment to actually unlock. So I actually think we’re pretty far ahead right now.

      Lauren: Well, and to your point, WallStreetBets is not just about, “I made a bunch of money,” it’s also people posting, “Shit, I just lost a bunch of money.”

      Anish: Though the subtext is: look at all the swagger I’ve got, I can lose all this money and it’s all good, you know.

      Lauren: Not always.

      Anish: Fair. Where this gets a lot more interesting is looking beyond social media and social networks and starting to talk about how this stuff drives an emergent set of products and how products are designed. Lauren and I have both talked about this, which is the concept that as a product, you can create value in a functional way, which is, “Hey, my credit score was X and now it’s X plus Y.” You can create value in a cognitive way, which is, “Hey, I now better understand my credit score,” or you can create value in an emotional way, which is, “I feel better about my credit score and my financial situation.” Historically, most products have been designed with a complete focus on the functional. And now we’re seeing the next generation, not just in fintech, but in consumer products that think more about the cognitive and emotional.

      There are also more offline examples than we’re all typically aware of. So one I learned about over the last few years is called ROSCAs, Rotating Savings and Credit Associations, which are these offline communities, mostly immigrant communities, that are managed by an individual. Everyone contributes, let’s say, $1,000 a month. And then each month if there are 10 members, one member receives $10,000. And typically these are folks in your community, you might meet them at church. It’s really hard to save $10,000, it’s a lot easier to contribute $1,000 a month. And then when you receive the lump sum, there’s always some big thing you want to do with the $10,000. There are tons of examples of these micro-communities that have not yet successfully been brought online. So, you know, not everything is starting from zero when it comes to digital products.

      D’Arcy: And those are interesting because there is a different iteration in every single culture and every single country.

      Anish: That’s right.

      D’Arcy: It is this robust offline behavior. And the question is, how do you bring it online? And how do you bring it online in a way that is culturally specific enough that it reflects the norms of that culture, but also in a way that’s scalable?

      Anish: So there’s the example of ROSCAs in a lot of communities all over the world, and then I think if you look at the flip of that, what’s the extreme San Francisco version? A lot of people here do things like invest in restaurants. Why would you ever invest in a restaurant? You’re probably not going to get your money back and there’s no liquidity. At best, it’s sort of cool to tell your friends maybe that you’re an investor there. Maybe you skip a reservation.

      D’Arcy: It goes to your emotional versus transactional. It’s not a transactional piece, it’s the emotional piece, right?

      Anish: Exactly. But the proof point of actually investing in something versus just frequenting something is very different. People want to participate, they want to express these preferences, and money is the strongest way to do so.

      Lauren: Well, and another example of something that’s inherently social—you’re investing in something that is then has a built-in social network.

      Anish: Exactly.

      D’Arcy: There’s also this amazing trend around fractional ownership. There’s a category of companies that includes Rally Rd., and Otis, and Mythic. They will take some asset—be it a classic car, be it a culturally significant item, be it a Magic card, be it a case of wine—and they will take that asset and they’ll functionally securitize it. And then you, as a user, can purchase shares of that asset. And in some cases, depending on the kind of investment that you make, you get certain levels of access or swag or other things that are associated with ownership.

      So on the one hand, you actually have a piece of equity, a share in something that is theoretically valuable because it’s a hard asset that has value. On the other side, you have this status of owner within this piece that is of value in a more emotional sense. You’re investing in cultural pieces, which may or may not be a good financial investment. But from an emotional/cognitive side it can be really, really rewarding. So I think that’s another version where this idea of social plus fintech is taking off.

      Anish: I love this example. And, you know, we’ve talked about this internally as perhaps the future of museums. I think that vision is really interesting, and it’s much more emotional than rational.

      Lauren: What’s the potential there? Are there areas where you see opportunity in some of these niche groups?

      D’Arcy: I think social and finance is like the holy grail, right? The social version of most products is the best version of most products. Engagement is higher, retention is higher, customer acquisition costs go down. All these things that most consumer fintech companies struggle with are solved by building the social product. To the extent that you can get something that threads that needle between social and fintech, it’s amazing, it’s magical, it’s incredible when it actually happens. It’s really hard to do, but when it does happen, it’s phenomenal.

      I think the biggest opportunity comes from finding the emergent behavior within niche groups at the social level, at the community level, and then figuring out how fintech or a transaction layers into or on top of that. The saying is “every company is eventually going to become a fintech company.” And I think that is probably the direction it goes, in which you have a weird social behavior that has some ability to layer a transaction inside of it. That’s how social plus money takes off.

      Anish: In my mind, the most direct way to start seeing this play out is just having more fintech products address emotional needs, as well as functional and cognitive needs. There are some fintech products like Joy, which is an app where you rate every transaction on how it made you feel. The goal of the game, of course, is to only spend money on things that make you feel good, which is kind of interesting. So I think that’s a product that’s completely designed around a set of emotional needs, with perhaps a set of functional outcomes as a happy side effect.

      I think there’s probably a middle ground where a lot of products that are focused on helping you buy your first home or reduce your debt or invest in stocks can actually start to design for these emotional needs when it comes to money. And that’s how we actually start to see this achieve scale.

      Lauren: Are there companies right now that you see making strides in that direction?

      Anish: I mean, I think an example of a company that’s really gotten this right is Credit Karma. And granted, I was at Credit Karma, but if you look at the tone of the emails, if you look at the ads that are on TV, if you look at the way the product is positioned, it plays as much to one’s curiosity and to taking some of the heaviness out of credit. And I think that’s been a really successful strategy for them. So I think this is a company that’s gotten it right when it comes to how you talk to your customer about these otherwise really heavy things.

      Lauren: And as people share more, it becomes less intimidating

      D’Arcy: Or if you can see yourself relative to other people. That’s the other way that Credit Karma works. It’s like, I know where I stand relative to other people. And maybe it makes me stressed or maybe it makes me feel more comfortable, but at least there’s some level of transparency.

      Lauren: Right. There’s some freedom in that transparency that perhaps is driving customer acquisition.

      Anish: That’s right. In terms of the products that have not worked, I think the product category that hasn’t really seen success is personal financial management tools. There’s two reasons. The first is that there’s a very small number of people who are super excited about budgeting and trying every budgeting app, which is why when a lot of these products launch, they get great growth in their first 18 to 24 months. You can get a couple of million users who are really engaged. That’s not actually representative of the wider market, where most people hate budgeting. And it’s not just because it’s a pain to keep a budget, it’s because it’s mostly bad news.

      So I look at a lot of these PFM and budgeting apps like calorie counting apps, they mostly make you feel bad and it’s easier to uninstall the app than it is to actually stick with the budget or the diet. So I think that’s a great example of a product category that, despite the fact that there’s real functional value there, it hasn’t taken off because it didn’t address the emotional challenge that the consumer is facing.

      D’Arcy: I think another category that has not worked super well is products that are designed to be social, but only transactional. So I think there’s been this long history of people trying to get people to be more public about what their portfolio is. And then other people can invest based off of that portfolio, and it benefits the portfolio manager who’s sharing it. That’s one where it’s an almost purely transactional relationship with purely financial incentives. And I think there’s been a lot of attempts at that. As far as I’m aware, none of them have really taken off. But I think that’s another category where when you just stick within one bucket, within the transactional side, it’s really hard to layer social into that.

      Lauren: So we agree that social meets fintech is really hard to do. But I’ve also heard you both say it’s the holy grail. Why is that? What makes it so powerful, if we can get there?

      Anish: I think if you just look at the most narrow lens, from a core business perspective— stickiness, cross-sell, acquisition—all of these things that are super hard problems for most fintech companies become dramatically easier if there’s a strong social layer. So that’s the most narrow lens.

      And then I think the broadest lens is ending this dynamic where we’re alone together. You know, everyone’s in a dark room feeling bad about their money with everyone else in that same dark room. And I think if you can turn the light on, all of a sudden it is an opportunity to uplift everyone a little bit and normalize the situation that folks are in. We talked about the good side of Instagam but Insta is also a very public place to talk about your spending. And I think that drives a sort of perverse set of expectations around what’s normal, and we should try to change that

      D’Arcy: Yes, there are multiple levels to why social plus money is this holy grail. Another lens is it broadens the solution space a founder can operate within, because now you’re not just on the transactional level or you’re not just on the emotional and cognitive level. You’re now across all three, if you actually have social plus finance or social plus fintech or whatever it is. So you can now design things that have some combination of those three levers. If you’re competing against a purely transactional thing or you’re competing against a purely emotional thing, you now just have more factors that you can operate across. The flip side of that is it’s combinatorially more complicated to do. But if you do it, you’re in a class of your own.

      Lauren: Thank you for joining us on the a16z Podcast.

      Anish: Thanks, Lauren. Thanks, D’Arcy.

      D’Arcy: Thanks, Anish.

      Anish: Cheers.

      • Anish Acharya is a general partner at a16z. Prior to joining the firm, he served as a GM at Credit Karma. He also founded SocialDeck (acquired by Google) and Snowball (acquired by Credit Karma).

      • D’Arcy Coolican is a deal partner at a16z where he focuses on marketplaces, social networks, and consumer technology companies. Prior to joining the firm, he co-founded Frank, a social lending platform.

      • Lauren Murrow is an editor at Future. She oversees posts, podcasts, & special projects for a16z's consumer and fintech teams. Previously, she was a senior editor at WIRED, where she edited op-eds and features.

      On Fear and Leadership — Product to Sales CTOs & CEOs

      Martin Casado, Armon Dadgar, and Sonal Chokshi

      There’s a few ontologies for describing the phases leaders — and their startups — go through, whether it’s product-sales-etc. or pioneer to settler. In any case, as companies evolve, so must the leaders — but can the same person transition across all these phases? When and when not; what are the qualities, criteria, and tradeoffs to be made?

      In this episode of the a16z Podcast, originally recorded as an internal hallway-style chat (pre pandemic!) a16z general partner Martin Casado, who co-founded but decided to remain CTO of Nicira — and previously shared his own journey, lessons learned, and advice for founders about bringing in an external CEO and the question of “to CTO or not to CTO” — and Armon Dadgar, co-founder (with Mitchell Hashimoto) and CTO of HashiCorp, chat with Sonal Chokshi about both managing their past psychology through these common questions and decisions. They also share their strategies on managing the specific tactics behind it all: Everything from the “dating” process of finding an external CEO to figuring out swim lanes; handling debates and decisions; who presents, who sells. And while the conversation is a brief glimpse into their longer personal journeys, there’s lessons in it for startups and leaders of all kinds on the art of hiring and sales, managing credit and conflict, and more…

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      • Armon Dadgar

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Crypto Creators: On Art Galleries to ‘Tokenized’ Collectibles

      Signe Pierce and Zoran Basich

      This episode features Q&As with two artists who are exploring crypto-powered auction sites and marketplaces – this is part of our ongoing series on the creator economy. The big picture is that emerging “tokenization” models, including non-fungible tokens, or NFTs, are creating new ways for collectors and investors to buy, sell, and trade digital art. More broadly, these innovations open the door to the tokenization of any products or collectibles that can be captured and owned digitally, and many new business models for creators.

      Marketplaces powered by NFTs open up new revenue streams for creators, because anytime digital work is resold or their tokens traded on these platforms, the creator automatically gets a percentage of those secondary sales. It’s all transparent and governed by code on the blockchain, and it’s a big shift in creator economies.

      Our first guest is one of the biggest names in crypto art, and one of the most mysterious. Pak is the artist and designer who created the AI-powered image sharing site Archillect. Pak has made it a policy to separate their personal identity from their online work, and prefers to keep their quote-unquote real identity hidden, so we conducted this interview by email and converted Pak’s answers to audio using text-to-speech software. As Pak has expressed in other interviews, it’s really the work that matters.

      And we do know a lot about the work, Pak has sold more than 60 pieces of digital art this year on the crypto-based auction site SuperRare, for more than $350,000. And that’s just one of the several platforms on which Pak’s work is sold.

      In this Q&A, Pak talks with a16z’s Zoran Basich about NFTs. These “non-fungible tokens” are unique assets that are not interchangeable. Dollar bills are fungible — each dollar bill is worth exactly the same as every other one. But works of art, for example, or any collectible, can be non-fungible — their value varies based on the market for that particular asset. With crypto, these assets carry digital ownership rights that can be easily exchanged.

      We start by discussing the whole concept of digital art.

      Why would someone pay a lot of money for something that seems like it could be easily copied? 

      PAK: You see, that’s a tricky question. Because the newcomer assumes that it can be copied but in reality, the collector of the NFT does not obtain a digital file, they get a unique and signed token that cannot be copied or owned by anyone else. So, assuming that NFT’s can be right-click-save-as copied is very similar to the assumption of going to the Louvre to take a picture of Mona Lisa to own it, or taking a picture of a plane ticket to copy it. NFT is not about the visible object, it’s about the permission and access to a thing.

      It’s only a matter of time until it becomes accepted in a wider sense. Every new medium for art had this struggle for acceptance. Having this struggle is not bad, it’s good. When the argument is over, this conflict and resolution will be the thing that will make crypto-based art valid. From my personal perspective, this is not a conflict or questioning of “is this art or not.” It’s more of a questioning of “is this unique enough or not.” And sooner or later it will be understood. 

      What was the dominant commercial model for artists in the past, and how might crypto change that? 

      I’ve never categorized myself as an artist even though a portion of what I do, design, surely touches art. Therefore, I believe, a better word for a “crypto-artist” should be “crypto-creator.” 

      From a design point of view, the options are working full time for studios, working as a freelancer, or having your own studio.  

      For crypto art it’s currently closer to the second one, the freelancer model, however, there is an important detail. A design client needs your work, but an art collector wants your work. Sometimes it feels better to be wanted than needed 

      What was your first exposure to crypto, and how did your interest develop? 

      I was able to meet bitcoin and make moves before its initial explosion. Being interested in value concepts as long as I can remember, I was instantly charmed by the idea of how value is transformed to this new exchangeable form. 

      NFTs, on the other hand, are new to me. The first time I met with NFTs was when CryptoKitties contacted me to create an Archillect kitty when they were forming. 

      Today, NFT’s are more than just little fun pieces. I believe in the technology it defends, therefore, I am happy to make waves in this branch of technological revolution, and evolution. 

      How do NFTs make it more attractive for creators to become crypto-creators? 

      It’s similar to cars becoming electric cars. “Crypto creator” is not a term that defines a special group of creators in my opinion. It’s only this now-new, soon-to-be-norm term for digital tradables. In other words, when it’s so easy to make it happen for any creator, why not?

      Most of the activity around crypto is financial — trading, borrowing, lending, and investing in cryptocurrencies. What are the characteristics of crypto that might hold appeal for artists, designers, and other creators? 

      Creation and destruction of value is always charming for any kind of creator.

      What advice would you give creators who are interested in exploring crypto as a new model for selling their work?

      Experiment!

      You have been working at the intersection of technology and art for some time. Apart from their possible financial advantages for artists, do crypto and the blockchain, or the idea of a decentralized technology in general, hold some deeper cultural meaning or symbolism, in your view? 

      Anything can carry symbolism, it depends on the receiver rather than the source. I value innovation, I value creation of value and I try to exist on that line, where something new pushes the limits of what’s widely known. Of course, decentralization of things holds a future of a new culture of technology, many new norms and standards, and many branches in unknown directions, and that’s exactly the reason to be there. Almost like the internet just before it went mainstream. 

      Let’s talk about revenue for artists. Will NFTs lead to a relatively small number of creators making good money, as has generally been the case in traditional art-business models, or do you think the distribution of wealth might become more equitable with crypto? 

      Art auctions are a completely different world, and it would not be fair to estimate NFTs based on that. Of course, NFT-based and supported art will have its own audience, and I expect to see similar dynamics with the traditional art world in terms of how things are evaluated. 

      On the other hand, NFTs can be used for many other things — this is what makes it powerful. Anything that can be digitally owned can be supported with an NFT. It has a lot of uses in media and entertainment, real estate, gaming, identification, or any kind of asset or collectible. It’s a technology that’s slowly going mainstream for art, but it’s not limited to that. Therefore, I do not think it will be only a small number of creators making good money. It’s going to be the base of many new business models of the future norm.  

      OUR NEXT GUEST is Signe Pierce, a visual, digital, and performance artist whose work has appeared in major galleries in Paris, Los Angeles, and New York. She’s currently featuring her artwork on the creator marketplace Foundation. On that site, the price of tokens associated with limited-edition works of art is something like you’d see on a stock market – the pricing is real-time and dynamic, fluctuating according to demand by buyers, who might be investors, collectors, or fans. (Signe recently opened NFT-based auctions of single-edition works as well.) Signe discusses why she went from working exclusively with galleries to trying crypto marketplaces, how this move affects her work and her business, and how crypto could change the way she engages with her fans. She also offers advice for creators interested in getting into the world of crypto.

      First she talks about how social media popularity several years ago opened her eyes to the idea of new monetization models for creators. 

      SIGNE PIERCE: I really had a big swing on Tumblr in the years 2014-2017 when Tumblr kind of popped off. And my pictures were just getting, like, hundreds of thousands of re-blogs.  

      And I just kept really returning to this idea of, why is there no valuation around this energy? And maybe how could there be? How could we turn Likes into money? How can I sort of flip the attention economy into an actual monetization form? 

      And that was what really got me thinking big about, yeah, I actually really can understand why this blockchain technology and tokenizing artwork could potentially be a really major disruption to the art world way of doing things as well as married to my vision for just a more prosperous world for artists and for people to enjoy art. 

      So how did you come to start working with Foundation?  

      I was put in touch with the curator, Lindsay Howard. We talked at length about our vision for wanting to find some new ways of doing things and make it so that artists have a little bit more stake in their work. I think partially because there’s a curator that I really trust who was inviting me in, as well as the fact that I could just sense that they had a finger on the pulse, all of that was what swayed me into wanting to work with them.  

      Which of your art is there now? Walk us through what that looks like and what’s being presented there at the moment. 

      So I have done a specific run of three works which are from what I call the Jangular Lilies series. I’ve offered two prints at editions of 100 and then one video at an edition of 10. I wanted to create a run that serves as this kind of cool, crypto training wheels for people who might be interested in this but that are nervous to get their feet wet. (Note: Foundation enables trading of limited-edition works via redeemable ERC20 tokens, along with its NFT-based auctions for single works.)

      Let’s see what happens, you know? And at the end of the day, you can experiment with the crypto. And it’s been really exciting to watch the trading aspect of it, when they trade in their tokens and they get to have the exchange. It’s just kind of been a new frontier for me personally. 

      And the interesting wrinkle is that people can come, fans, collectors, and they can buy your artwork at the price at which it’s listed. And when you are ready to produce that artwork they will get a physical copy of it, right? 

      Yes. 

      Or, while they’re waiting for you to complete the artwork and fulfill that order, they can also trade the token that is associated with that artwork. And so the price of that artwork is going up and down in value, or it’s dynamic, right? It can go up and down. 

      Exactly. 

      So that goes to the point of what you were saying about people being able to trade it. And that’s really what makes it interesting here because now you’re talking about not this static price but rather a dynamic price that can lead more people to perhaps get involved with this from sort of a purely trading or investment aspect on top of the collectible or fan aspect of it. 

      I know not all artists are gonna like this, and not all artists are going to find this to be their jam. But I see this as kind of an interesting way to think about our works like stock, in a way. I’m interested in watching the way that that’s playing out in this sphere because it’s new and it needs to be pioneered in order for it to become normalized. 

      So from the buyer’s perspective or from the collector’s perspective, they can wait for the price to go higher and trade it and make a profit on that and not end up owning your physical piece of art, correct? 

      Yes, exactly. And I think that’s kind of also a cool way for people to support the artist. You might not necessarily end up with a print if you don’t decide to redeem your token. You’re still supporting me, the artist, in a little way just by engaging with the trading. And I think that’s, again, a cool way to get people into the world of crypto and supporting the arts.  

      And one thing we should mention, you just touched on it, is that every time anyone trades one of these tokens you receive a cut of that. The artist receives a cut of that. 

      Exactly. That, plus the fact that as the editions decrease, the more editions that are sold, the price increases. And that is also a really exciting aspect of this.  

      And how do you think about pricing? How do you decide what the initial price will be? And how big the limited edition will be? 

      I’ve spent a lot of time thinking about this. I’m going to kind of keep my works that are dealt through gallery systems and the art market, I’m going to purposefully keep them low and, you know, rare. They come with the full package of framing and serious fabrication. With these works, they’re high quality, digital C prints. It’s just larger editions, and therefore the price kind of matches the fact that there isn’t as much scarcity attached to them. 

      But again, I’m finding there’s a sweet spot for me wherein I’m still able to work with the fine art market, but I’m also able to make it so that the work is accessible to more people. I think my work is valuable. I take it very seriously, and I want it to be valued appropriately. And I want it to appreciate over time. And I think the fine art market finesses that opportunity. 

      But I don’t want it to be completely forbidding to everyone else in the world because that goes against my general ethos of art being for the people. So to me that’s my personal approach. Again, not everyone’s approach, but this is my way of making it so that we can kind of play both fields. I’m not giving up on the fine art market or the fine art world, but I’m also really interested in trying new things and inviting more people into that, into the opportunity to become a collector. 

      So when you go to the site and your artworks are there, it’s like a stock market, right? There’s a price, and it shows how much it’s risen or fallen. I think I looked this morning and all of yours were up over 100%. Congratulations. 

      Yeah, we’re doing pretty good.  

      How often do you look at that? And what’s that like? It’s just a whole different kind of insight. It’s a real-time look at how people are responding to your work, right? 

      Absolutely, and that’s exciting. It feels very modern. I’ll be super honest and say I’m not, like, a finance guy, you know? I’m an artist. But I’m still, you know, I’m a modern artist, and I really, I’m interested in modernity and the future as well as looking at the way it works, understanding it in order to better think about it. So when I get to see in real time the valuation of the work, that’s kinda thrilling because, again, I’m so used to this kinda Tumblr culture where you get to see your numbers going up. You’re watching Likes and re-blogs happen. But there’s nothing actually attached to it for me other than minor dopamine bursts. So this is actually creating a little bit of capital to it to make it a little more fun at least, rewarding.

      Is it an experiment where, yeah, you might make a little bit of cash? Is it a significant amount of cash that’s possible? Like, how much of a revenue stream does this represent for you? 

      At the end of the day, I’m making money making my work, and that’s exciting. And there is profit. I’m making a profit from this, and it’s another avenue for revenue for me to make my art and get it out to the people. To me these are all wins. While the profits are not as gigantic as if it was on, you know, the fine art market immediately, but that’s for one work, you know? This is multiple works that, over time, it is similar numbers, you know? In fact, sometimes more. 

      So maybe it doesn’t make that much sense for me to only be having, you know, extremely high prices, and there’s only one period, and then you have to wait for that appreciation and valuation. And then if it flips on the secondary market, I don’t get to really see, I often don’t get to see that unless I’ve contractually negotiated it.  

      How do you think about digital art versus physical art? Because I think in this case right now you are gonna produce physical pieces that you’re gonna send to people. There’s, like, a reproduction cost involved there, which presumably cuts into your profit, whereas digital work is more easily reproduced. 

      Totally. 

      But then also, digital art historically has been hard because, like, how do you stop it from being copied? But with crypto there’s this kind of additional thing of the token, where you know the provenance, and you know it’s this unique piece of art that cannot really be owned in that same way. So how do you think about those things as part of a business model? 

      That’s a really great and important point/question. And I think that people are and have been struggling with getting their head around the idea of, like, paying for a video or paying for a GIF. And how can it have value if it’s not tangible, if it’s not a physical artwork? And it’s taken a lot of thinking and pioneering from a lot of artists and, you know, technologists. And to me, like you said, the provenance of blockchain technology, which guarantees authenticity of the work, is essential to make this possible. 

      Once we can kinda get the collective consciousness head around that, I think it could be really revolutionary for how art continues to be made and traded and collected. 

      I want to talk about fan engagement too because this opens up really interesting possibilities. And it’s happening now with your artwork where people are engaging with it in a financial way, but tokens also potentially enable other kinds of engagement as well. People can become “super fans” or have access to certain things in your life or in your work that non-token holders may not have. So have you thought much about the evolution of fan engagement as it relates to crypto? 

      I think that there’s totally so much potential for this. Accessibility, the access points are what’s gonna be a really big part of that fan engagement generation. There’s all kinds of different ways to approach this. 

      I just want to do it in a way that I think is actually cool. That’s honestly one of my big things about what I’m trying to design is, like, I want it to work in a way that I would actually want to do it. I’m interested in figuring out how we can work with fan engagement in a way that I really would feel comfortable asking my fanbase to participate in, when I kinda, like, flip the script and I think about an artist that I love, what would I be interested in paying to gain more access to their work? And that really is kind of the decision of the artist to configure what the different tiers of value are. 

      So I’m interested in your perspective on what artists who are kind of interested in this, who are trying to break free of this rut, this traditional kind of commercial system around art, what should they know as they look to explore crypto? And how much do they need to know about crypto before they get started? 

      I think it’s always helpful to have someone hold your hand a little bit. If you don’t have a natural direct resource to introduce you to this, throw yourself into some research and teach yourself about it. I mean, I’ve really had to do that for myself with this. I barely scratched the surface of fully understanding how all of this works because, again, I’m an artist. So it’s not necessarily my 9-to-5 attention span to be reading about advanced blockchain technologies. But, because I’m an artist who’s interested in future models and methods, it’s important for me to sit and focus on these ideas to get it into my head. So that’s part of it. 

      But another part of it I think is just this fearless entrepreneurial spirit of, what do I have to lose? If you’re currently not making much money or any money wheeling and dealing your art, what do you have to lose but to throw yourself into something new and see if it sticks, you know? And I think that’s a really important energy for people to hold in general is just this kind of, “Let’s go. Let’s try something. Let’s try something new.” And if enough people host that innovative spirit, that’s when things start to crackle and spark and change.  

      Awesome Signe. Thanks so much for being with us. It’s been a pleasure to talk to you. 

      Absolutely. I had a great time. Thank you for this opportunity. 

      Featured image: Pak, “The Balance”

      • Signe Pierce

      • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

      The Great Data Debate

      Bob Muglia, Michelle Ufford, Martin Casado, Tristan Handy, and George Fraser

      Lakes v. warehouses, analytics v. AI/ML, SQL v. everything else… As the technical capabilities of data lakes and data warehouses converge, are the separate tools and teams that run AI/ML and analytics converging as well?

      In this podcast, originally recorded as part of Fivetran’s Modern Data Stack conference, five leaders in data infrastructure debate that question: a16z general partner and pioneer of software defined networking Martin Casado, former CEO of Snowflake Bob Muglia; Michelle Ufford, founder and CEO of Noteable; Tristan Hardy, founder of Fishtown Analytics and leader of the open source project dbt, and Fivetran founder George Fraser.

      Their conversation covers the future of data lakes, the new use cases for the modern data stack, data mesh and whether decentralization of teams and tools is the future, and how low we actually need to go with latency. And while the topic of debate is the modern data stack, the themes and differing perspectives hit on an even more longstanding question: how does technology evolve in complex enterprise environments?

      Show Notes

      • The future of data lakes [1:07] and specific operations that may impact their usefulness [6:01], including AI/ML [8:55]
      • The evolution of two-stack architecture [9:35] and Arrow as a potential solution [11:32]
      • The pros and cons of a data mesh [16:18], future use cases for the modern data stack [20:07], and data apps [22:05]
      • Discussion of latency and ways to reduce it [22:46], and predictions for a future data platform [25:41]

      Transcript

      The future of the data lake

      George: I’m going to kick this off with a spicy topic, at least spicy in this crowd, which is data lakes. Data lakes is a blurry term used by different people to mean different things, but for the purposes of this discussion, let’s define data lakes as tabular data – so tables, rows and columns – stored in an open source file format, like Parquet or ORC, in a public cloud object storage, like S3 or Google Cloud storage.

      In a world where we have data warehouses that use object storage to store their data and give you some of the advantages of data lakes, do data lakes still have a place? Let’s start with you, Martin, does the data lake have a future?

      Martin: One of the biggest fallacies that we do as an industry is we look at an architecture, and we’re like, oh, that can do all of these things, therefore it will be pushed into service to do all of these things. And that’s just not how technology evolves. We make decisions in the design space based on the primary use cases that technology is being used for.

      If you look at the use cases that data warehouses are being used for, they’re largely driven by analytics, which is a certain workflow, it’s a certain query pattern. And if you look at data lakes, it’s actually quite different. They tend to have more unstructured data, focused on operational AI, compute intensive. If you look at the respective technologies, they’re just being optimized in this massive design space for different use cases.

      Architecturally, sure, they can both do what the other one does, but in the end, you’ve got products and companies optimized around use cases. And I think the operational AI use case is the larger one, and it’s growing faster. So I actually think over time you can argue that it’s the data lake that ends up consuming everything, not the data warehouse.

      George: You’re just trying to provoke Bob there, Martin.

      Bob: You succeeded.

      Martin: I’m watching Bob’s face.

      George: All right, Bob. Let’s hear from you. The data lake, does it have a future?

      Bob: No, I see these things very largely converging onto a relational SQL-based model. Five years from now data is going to sit behind a SQL prompt, and SQL data warehouses will replace data lakes.

      From the perspective of storing structured and semi-structured data, the cloud SQL data warehouses already do everything that is necessary, and there really is no reason for people to have a separate data lake except for historical precedent. A lot of companies come from environments where they had a lot of semi-structured data in a Hadoop environment, and having a data lake is a natural transition. And in a sense, the data lake, which is really S3 storage together with any tools you want to put on top of it, is a very generalized platform.

      But, over time, infrastructure evolves to take on more and more of the use cases. SQL relational data warehouses have evolved to the point that for structured and semi-structured data, storage and query, they subsume pretty much all of what needs to be done today. What remains is images, video, documents, PDFs.

      Now I don’t call that unstructured data. I think that’s a misnomer. There is no such thing as unstructured data. All data has structure of some kind. Structured data is tables, rows and columns. Semi-structured data is like JSON. It’s hierarchical in its nature. And I think there’s a third category of data, which is what I call complex data: images, documents, videos. Most things that are streaming fall into this category, and more and more machine learning can be applied to the content of those data sources that turn it into semi-structured data that can be used for building complex data applications and for doing predictive analytics.

      So what’s missing in the case of the data warehouse today is the support for complex data. But that’s going to come. That’s called a feature. Can you imagine if you could transact, fully transact all of these types of images, videos, and things together with any source of semi-structured data in a data warehouse? The applications that open up are remarkable, and that’s going to come in the next two to three years.

      Michelle: I could see images being easily retrieved from the database. But do you actually see all of the image processing or the video processing taking place in the database as well?

      Bob: Not with SQL. SQL can’t do that. You’ll use procedural logic and Python, or something else to do that, at least for now. In the long run, relational will win, too, but that’s probably more like 8 to 10 years away.

      Martin: I think we’ve been waiting for that for 40 years, Bob.

      Bob: We have, but look what’s happened. Over time, navigational and hierarchical in the 1980s was replaced with SQL. OLAP was replaced with SQL over the last 10 years or so. We’ve replaced MapReduce with relational. All of these things, relational always wins.

      Michelle: Well relational wins for the actual retrieval, but what about the processing? The technology that you need to process images is fundamentally different than you do to retrieve data records.

      George: Tristan, what are your thoughts on this?

      Tristan: So, I completely agree that SQL is going to dominate data processing, at least a very large chunk of data processing, but there’s different APIs that the data lake and the data warehouse expose. There’s the file storage layer, and for a lot of reasons I believe that an organization will store their files one time. You will not have a data warehouse copy of the file and a data lake copy of the file, which, in some architectures today, that’s what you see. And that requires you to have an open source file format that is shared between your data warehouse use cases and your other use cases.

      Above that you have indexing and meta data that is a core part of the data warehouse, but it’s also a core part of the data lake. I think those have to also start to converge so that different use cases can take advantage of the same stuff. And then you have the SQL prompt, and maybe, at the SQL prompt layer, the data warehouse dominates, but I think you need to allow different access patterns as well because one closed source firm is never going to accomplish literally all data processing use cases in the world.

      Bob: All of these things should interoperate in an open source and an open format way. But the issues of format have kind of gone away because you can input and output any kind of format and export into any kind of format very easily.

      The question is: what are the operations that actually need to be performed against data that sits in a data lake? Today anything associated with complex data, the data warehouse can’t help you, and so there’s a huge reason to have a data lake today. In 2025, I don’t think so.

      I think that we really have five platforms being created globally: Snowflake, Databricks, and then the three clouds. Both Snowflake and Databricks, while they will come from very different places – Snowflake will always be SQL and declarative in its approach, and Databricks certainly historically has been procedural and code-based, so it’s a version of SQL versus code in some sense – you’ll see both companies and pretty much everybody else in the industry offering both within their platforms.

      Martin: So, you’ve got two technologies that start with different use cases, somewhat different architectures, but they’re clearly going to a converged point, which is you have some declarative something, and you have some procedural something. Whether one is on top of the other at the end of the day, they can both do both. But, in the meantime, you have this decade-long journey, and in that decade-long journey, there is an architecture that’s optimized around use cases. The amount of tradeoffs and decisions you make when building one of these systems is…

      Tristan: Yeah, like TimescaleDB has very different characteristics than Snowflake, and they are characteristics that are optimized for workflow.

      Martin: Yeah, entire companies focusing on different points in the design space with different optimization parameters. It’s the use case that drives the technology because of all of the gravity around it. And so, again, if it turns out that AI/ML and an operational use is growing quicker, which it seems to be, that is going to dictate the technology from an architectural standpoint.

      Tristan: Martin, you’ve said a couple times now that the AI/ML space is appearing to grow faster. I’ve actually not heard that assertion before.

      Martin: Let me clarify. So broadly, there are two use cases. There’s the analytics use case, which is driven by queries and dashboarding. The other one is creating a complex model from a data scientist and then serving that in production. That does things like wait time prediction. That does things like fraud detection. That does things like dynamic pricing. These were folks in R building complex models on existing data and then coming up with a bespoke way of serving that. That is very clearly now turning into a pattern that’s being served by a data lake.

      Now it’s on a much smaller base, but if you actually look in the industry, it’s a very rapidly growing use case.

      George: Michelle, you’ve spent time in both the data science community and the analytics community, and notebooks in many ways are the place where these things sometimes come together. I’m curious to hear your thoughts about how the two stacks have evolved. Maybe they’re converging. Maybe they’re building each other’s features and getting more similar, but where does that take us? Do we still have two stacks five years hence?

      Michelle: I think we’re going to continue to see greater and greater specialization because we’re not going to have the ability or the budget to hire enough data scientists. Those stacks are going to continue to evolve, and it’s going to be specialized based upon what it is that they’re trying to do.

      The data lake will have a place. Your images, your blob storage, all of those things are probably going to remain in the data lake and have a home there for a long time to come. I just think it’s not going to look like how it looks today. Today, it’s just been a lack of understanding around what data do we really need to collect? We went from one extreme to the other. We weren’t collecting any data. Now we’re collecting everything because we don’t know what’s valuable. And the reality is that’s not necessarily a good idea either.

      The movement of data, I think we’re going to see that stop, but format is going to be really important. We need that interoperability because reprocessing data at scale is just cost prohibitive. It’s time prohibitive. It’s not something we want to do if we can avoid it.

      And I think you’re going to see decentralization here, at the lower levels, where you’ve got either the business units embedded, or you’ve got your new product teams, you’ve got your data science teams embedded in those product teams. You’re going to need a unifying layer at the very top the form of technologies that make it easier for everybody to query or be able to serve information.

      I think that the notebook is probably the best suited for that because it does have the language agnostic approach. It gives you the ability to look at both data and code and have all of that context, that rich business context, the visualizations. We’re going to see that evolve as this modern data document, and we can use that as part of our unifying layer because your data scientists can then work with R, your data analysts can work with SQL, but we can, at the end of the day, really hide all of the code and really get to: what is the business implication of these things that we’re doing?

      Will two stacks become one?

      George: This really brings us to the second major topic that I wanted to discuss, which is: how do we bring the machine learning, Python, Scala world, and the analytics, SQL, BI tool world together? There really are two stacks and two communities who sync the exact same data sources to Delta Lake and to Snowflake simply for operational reasons. There’s not a fundamental technological reason, but it’s just the way the tooling has evolved. It’s too inconvenient to cross that boundary.

      And there’s essentially three visions of that world. One is that you’re going to put machine learning into SQL, and probably BigQuery is the furthest along in pursuing this. You basically create a bunch of UDFs that do your linear algebra stuff. The other is more the Databricks vision where you put SQL into Python or SQL into Scala and you use data frames to do that. And then there’s maybe a third vision where you use Arrow, the interchange format, and everything can just talk to each other, and you can arrange it any way you want.

      Which of these visions do you think is going to win?

      Michelle: What I would like to see win is something like Arrow, so that you have the interoparability. You’re going to see machine learning moving into SQL because you’re going to have data engineers who are perfectly capable and have the need to do some anomaly detection or some logistic regression, and it’s within their ability to do that. Feature engineering is just another data transformation for them. But they don’t have the same background in stats, and so they can only take it so far.

      And then you’re going to see, on the other side of the spectrum, your data scientists where they have all of this really great math background, and they understand how to do more advanced deep learning, but they don’t have the technology skills. SQL is the most successful language for working with data, so you’re really going to see both of them really become capable of supporting both use cases. Ultimately, you’ll continue to see specialization where the things that you want to do if you’re trying to do deep learning are just fundamentally different than the types of things if you’re just trying to do predictive models.

      Tristan: I think a lot about the Arrow vision of the world, and I think that will end up in the fullness of time dominating for the same reason that Martin has been talking about: tools end up evolving to the personas that they serve and the use cases they serve.

      I want to do all the data prep and feature engineering. And then I want machine learning models to be trained on top of that. People do that, certainly. But the fact that the infrastructures to do those two different things are generally separate creates this big slowness. It’s purely a technical slowness. Arrow doesn’t solve all of that. Arrow certainly helps, but, there’s dumb things like the servers that do those things are in different clouds. And the interchange fees, what do you, do you call them interchange fees?

      George: Egress fees.

      Tristan: Egress fees are expensive.

      George: They’re criminal. They’re not just expensive. They’re ridiculous.

      Tristan: As more people do this, it’s going to be become smoother. they’re going to become more localized.

      Martin: There’s a reason why you’ve got multiple languages, and it’s not because one is Turing complete and the other isn’t. The reason is because people build their entire workflow around languages and all of the tools, and so you’re going to have a heterogenous, fragmented system. Therefore you do need to have open interfaces.

      George: Bob?

      Bob: I’m a big believer, at this time, in the approach of having multiple systems that interact with common formats.

      Arrow is a huge step forward for that, not just because it’s an efficient format, but because it provides a consistent in-memory layout for people to do advanced analytics in their Spark environments. It’s the way the world is working right now because most customers actually have a data warehouse and an analytics platform separately, and they are connecting them together.

      Now, I’m going to continue to be the ultimate radical, however, and declare that the approach that we’re taking today in terms of machine learning is still roughly the approach of the internal combustion engine in the automobile. The approach that’s happening where Arrow ties together those predictive systems with declarative databases, that’s really the creation of the hybrid, or the Prius era.

      Hybrid will dominate for the next, say, three to five years. You will see hybrid systems being built by every major vendor, and all of them will have a full predictive stack and a full declarative, relational, SQL stack built in using some kind of interface like that. But that’s only until relational actually solves the broader set of problems.

      George: Does that mean that you’ll be using SQL functions, PredictX, or…?

      Bob: No. Ironically, I think that while SQL will dominate well into the 2030s for doing data modelling and data transformation, there’s another step beyond that which is business modelling, and that needs to be represented in a knowledge graph. Knowledge graphs are how we’ll do predictive analytics in the 2030s. And what needs to happen is a whole new generation of data system that’s based on relational knowledge graphs to create that.

      Data mesh: decentralized teams, unified architecture?

      George: Michelle, you brought up a term earlier that I wanted to follow up on, which is data mesh. And I wonder if you could define that briefly for everyone because similar to data lakes versus data warehouses, there’s a question whether going forward that’s more of a historical phenomenon or an actual, good architecture that we want to continue.

      Michelle: Data mesh is really a concept of decentralizing the data processing and the ETL and the analytics into each individual business unit and then having some sort of unifying solution at the top. To do this well requires having specialized data teams, having specialized roles, having infrastructure as a service available to them for data processing, and then having some overarching standards board, almost like a federation, of your data engineers to ensure that all of your ETL looks consistent so that as you are trying to do data retrieval on some common, query tool, you’ll have that familiarity that you need.

      We are going to see things like Arrow really come to the forefront sooner rather than later. I think customers are going to demand it because of all the challenges that we’re currently having. You’ve got all of the cost of the storage and the processing. Your teams that are trying to do the processing don’t have the business context that they need. As a result, you have this back and forth and a lot of wasted time. You’ve got a lot of data quality errors. You have data multiple times. Ultimately, we want to take that body of knowledge and put the technology where that body of knowledge lives. The data mesh is an attempt to do that.

      Bob: One part of what the data mesh folks are talking about is how to organize and how to structure a team to manage data across a large enterprise with very disparate and important data sources. That’s very, very important, and there’s some good ideas in data mesh for that.

      Architecturally, data mesh has this sort of odd idea that data is basically streaming, and you can use facilities, like Kafka, to do transforms as the data is in flight. And I don’t believe that.

      While there is streaming data, and you can do quite a bit with data that’s simply streaming — in other words, append-only data — to me, another critical source of data is transactional data coming out of business systems. The streaming solutions have no answer for that, and they just pretend that data consistency is unimportant. I don’t understand that because I put data consistency at the top of the issues that I think about when I think about managing data.

      Martin: Mesh has historically been one of these terms that conflate architecture with administrative domains, and distant service mesh, and distant Wi-Fi mesh, and mesh networking, etc. I think actually Bob is exactly right, which is there is a very real issue with separate administration domains, separate processing domains, separate access to tool sets. That’s very, very different than building a fully distributed architecture, which just tends to be hard and inefficient. And it’s often not the people that promote the mesh idea, but when people hear the term mesh, they default to full distribution, which tends to be just a bad way to build systems.

      George: Said like a networking guy.

      Martin: Having seen this exact same thing happen in other domains for a couple of decades.

      Tristan: All of us are very technology-focused human beings, so when we think about data mesh, we tend to think about the architecture part of it. Bob, I’m glad you pointed out the distributed teams and the people aspect of this. My constant question for data mesh is: why can’t you enable the distributed nature of what you’re talking about with a unified architecture?

      Michelle: My preference is always to have one data set that is very clean and well understood that we do not have to move anywhere, that is performant alongside our large batch analytical processing, which is also working with our data science. That’s the nirvana. That’s the goal is to just have one data storage and then having something that sits over top of it, and each of those different things are specialized for each of the different use cases but you have one data store.

      Next use case for the modern data stack

      George: The modern data stack keeps swallowing up more and more use cases. It killed cubes a while ago. It’s mostly killed Hadoop at this point. It keeps pulling more use cases into its orbit because it’s fundamentally so flexible and so capable of doing many different things well enough that you don’t really want to buy another system, build another system for one use case. What are some of the most interesting, surprising, significant use cases that may start to get pulled into the orbit of the modern data stack in the next couple years?

      Bob: Complex data. We now have all this very, very interesting stuff that’s happening in predictive analytics. And to me we’ve gone from semi-structured data as being the most interesting data sources to now having a wide variety of data sources. I was talking to a company involved in the medical field yesterday, and just the rich amount of data that exists, and the images, and the doctors’ notes, all of that is opaque to our systems today. It will not be in five years. That will all become part of the modern data stack, and to me that’s a gigantic transformation for the types of applications that will be created in the years to come.

      Tristan: My last job was I ran marketing for a company, and I was deep into growth marketing. The problem that you run into there is that you’re constantly writing code to push data back and forth between systems because the different operational systems do different things, and you need the same data in all of them.

      No one has yet rearchitected the systems to, in the modern data stack, just take all of the work that you’ve ingested and now push it back out to your operating systems or your operational systems. But I think we’re at the beginning of that.

      Bob: What you’re really talking about Tristan is the advent of the modern data app, which basically is an operational application that autonomously can make decisions for the business. We’ve seen very few of those and very trivial examples, but boy will they be significant in the future.

      George: There’s really two visions of the data app that I’ve seen. One of them is the data app is a separate system, and you take the important data from your data warehouse, and you push it. Then the other vision is the data app is just natively built to run on top of the data warehouse. I’m curious whether people have opinions about those two models and where they see that going.

      Bob: It’s really the same conversation we’ve been having about how these things are built. A data app is predictive analytics that actually takes autonomous action. It takes the data that would otherwise be presented to a person and instead leverages that to actually take actions within the business. They’re being built every which way today because there are few good tools to build data apps. That will not be true in a few years.

      Latency: How low do we need to go?

      George: One of the things that you run into when you try to build data applications and take action automatically is latency becomes incredibly important. Everybody in the ecosystem is battling this right now. I think there’s a lot of different visions of how we’re going to crush the latency problem and how low we need it to get. How low does the latency need to be? At what point do we have most of the interesting use cases

      Bob: People have dozens to hundreds or even thousands of operational systems. More and more, they’re SaaS operations. They’re outside of your organization. They’re always a source of truth now. They are the present, and a data warehouse or a data lake is about historical or the past.

      What does that latency need to be? Does it need to be zero seconds? I don’t think so. There are applications where zero seconds or instant is required, mostly having to do with eventing and alerting of some sort. Most of the time, if you can get it in a minute or two, you can leverage that data inside your historical system with predictive analytics to begin to perform actions on it.

      Martin: This is a very complicated topic that I think is very use case specific. But there tends to be serious trade-offs that systems designers make between latency and throughput. If you want higher throughput, you batch. And the reason that you batch is that you don’t have as many domain crossings.

      However, if you look at most systems, you can make the tradeoff. Meaning you could do low latency in a data lake, and you could do high throughput in a data warehouse, or vice versa. These are not architectural limitations. They just tend to be the tradeoffs that were made as a result of serving whatever the primary use case is. I’ve heard a number of these latency-throughput tradeoff discussions, and you actually get down to a machine level, they are just a result of the tradeoffs that were made on the system going into it.

      George: One of the interesting things that we see is that the point at which you start to have to spend a lot more to get the latency lower is actually lower than people think. I suspect you can get down into the 10 second range with the throughput optimized architecture. Basically, the throughput optimized architecture I suspect will go lower than we expect.

      Michelle: What do you imagine will happen with the serving layer? Your website still needs to operate over that data. Are you imagining that there’s just going to continue to be a caching layer? Or is that going to be a separate system?

      Bob: It depends on what the characteristics of the system need to be. If something needs to be really low latency, today’s data warehouses are not always the right solution for it. It just depends on the application. Latencies will go down in these products, but to Martin’s point, some of the architectural choices make the latency characteristics of a Snowflake somewhat different than, for example, the latency characteristics of a MemSQL.

      Tristan: One of the things that I would like to see more of in the future is Lambda architectures, but with off-the-shelf tools. So my data is flowing into a more streaming-like system and a more batch-like system so that I can get the best of both worlds. You’re making tradeoffs when you build these systems. As a user, I want to be able to choose and have both of them.

      George: Well, we have one minute left. I’d like to ask a yes or no question for everyone: will there emerge another major data platform alongside Snowflake, Databricks, Google, AWS, and Azure? We’ll start with you, Michelle. Yes or no?

      Michelle: Yes.

      George: Bob?

      Bob: What’s your timescale?

      George: In the next five years.

      Bob: Yes.

      Michelle: Yes.

      Bob: But the new one may be relatively small relative to those guys.

      George: Well I said major. That sounds like an in-between…

      Bob: Snowflake was small five years ago.

      George: Tristan?

      Tristan: I think no.

      George: Martin?

      Martin: Yes.

      George: All right. Thank you very much, everyone, for joining. This has been a really fun conversation. I really appreciate all of you being here. I know our audience does as well.

      • Bob Muglia

      • Michelle Ufford

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      • Tristan Handy

      • George Fraser is co-founder and CEO of Fivetran.

      How to Moderate Talks, Panels, Meetings, and More (Virtual and Beyond)

      Matt Abrahams and Sonal Chokshi

      [tocmobile-embed]

      How to moderate good, productive discussions and navigate tricky conversations is top of mind — whether doing a panel, conducting a live event, presenting a talk (or even hosting a podcast), managing (or just participating in!) a meeting. Especially in a world where remote and virtual work is increasingly become the norm for many knowledge workers (given the pandemic and even beyond) — one in which we’re increasingly communicating through little “Hollywood Squares, Brady Bunch”-like boxes.

      So how to translate physical and nonverbal presence in such virtual environments, or voice-only modes? How to manage unruly discussions? Do parasocial vs. social interactions change things? And beyond these broader contexts, how do the things inside us — whether agendas, tics, anxiety — manifest outwardly, and can we better control them?

      In this episode of the a16z Podcast, Matt Abrahams — lecturer at Stanford’s Graduate School of Business (where he also has a podcast, “Think Fast Talk Smart”); principal and co-founder of Bold Echo (a company that helps people with presentation and communication skills); and author of Speaking Up Without Freaking Out — shares frameworks and best practices, in conversation with Sonal Chokshi. The discussion offers many concrete tips for moderation and communication for anyone, across all kinds of mediums and modes.

      image: Paul Hudson / Flickr

      Show Notes

      • The importance of planning when moderating an event [1:45], and how to deal with unruly discussions [3:37]
      • Using paraphrasing to guide the conversation [5:21] and techniques for bridging and linking ideas [7:41]
      • Further discussion of preparation [12:50], setting event ground rules [20:07], and how to be the voice of the audience [26:40]
      • All about virtual communication [27:56] — visual issues [29:22], vocal issues [31:47], and reducing verbal tics [33:34]
      • Managing anxiety through breath control [39:20]
      • Discussion of new social audio platforms, such as Clubhouse [46:34]
      • How to structure a talk [49:12], and best practices for introductions and conclusions [52:02]
      • Concrete tips for reducing speaker anxiety [56:52]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal — and I’m here today with an episode all about one thing (but also many things), which is, How. To. Moderate. And I don’t mean moderate in life, like “everything in moderation”; I mean it in the sense of moderating when you’re speaking — whether managing or participating in a meeting, presenting a talk, speaking on a panel or live discussion, even doing a podcast, and more.

      We especially go deep on something that’s top of mind right now given the pandemic — which is that many knowledge workers, who have the privilege and ability to work from home — are now working and communicating entirely online and virtually, and many will probably continue to do so well beyond. So how does that change moderation? Where do the differences between in-person and remote — as well as the evolution of tech & tools — come in?

      Our special guest for this episode is Matt Abrahams, who’s a lecturer on strategic communication and virtual communication at Stanford’s Graduate School of Business, where he also has a podcast called “Think Fast Talk Smart”; he’s the principal and co-founder of Bold Echo, a firm that help executives (and anyone, really) who wants to improve their communication, learn new skills, or just improve upon and sharpen their existing skills.

      Our conversation offers frameworks — and lots of concrete tips — for moderation all kinds of modes and mediums, including covering how to manage unruly discussions, how to prep (and the tensions between being scripted vs not); how to manage tics; how to translate physical and nonverbal presence, even in virtual environments; differences between parasocial and social interactions, does that change things?; tips for managing speaker anxiety; and how to structure a panel, talk, or discussion from intro to conclusion. But we begin with the role of pre-work (and post-work) around all kinds of conversations.

      The importance of planning

      Matt: As somebody planning a communication interaction — be it online or in person — you need to think about the *things you do in advance of it happening, *what you do during, and *what you do after.

      So in terms of what you do in advance: You’re figuring out who your audience is, what’s important to them? What themes do you want to get across as part of this communication? What’s your goal — and to me a goal is very specific, a goal is about *information, *emotion, and *action: *What do you want people to know; *how do you want them to feel; *what do you want them to do? Are there ground rules you want to establish?

      In the midst of moderation, when it’s actually going on: Your biggest skill sets are: *your ability to listen, *your ability to paraphrase, and *link and bridge ideas. That’s what helps a smooth interaction take place.

      At the back end, when it’s over: You know just because the interaction has ended (the meeting is over, the presentation is over, whatever) — you then have to think about how do I follow that up; and how do I make sure the information is acted upon; and set myself up, and the others, for success for the next interaction.

      So it is a process that starts way before people ever enter into the call or the room, and it continues long after they’ve left.

      Sonal: So, what’s the difference then between sort of planned meetings — like presentations and panels — versus spontaneous, more organic sessions.

      Matt: The preparation piece I think is the same, but as it’s going on, if it is a free-flowing activity — maybe a brainstorming meeting, a feedback session — your job as a moderator is really to just guide and steer it in the direction that the participants are taking it.

      In a more formal situation — like a panel, or a decision-making meeting — you have to be much more directive: You have to keep things on track; you have to be monitoring the agenda, and the time, and the different types of contribution.

      Managing unruly discussions

      There might be power dynamics at play: It may be the case that somebody is acting the way they’re acting, because they have additional information that they can’t share. It may be that the person had a bad interaction before they came into the situation. So it’s also very important to — while moderating, while facilitating — to take a step back and try to understand at a meta level, what’s going on in the interaction, and perhaps decide to act on it — give some direct feedback or guidance — or perhaps pull back, and do some of that either on the side… or later.

      Sonal: It’s so fascinating, because there’s a psychological component here, which is, it’s the difference between whether you go into an interaction — any kind, whether one-on-one, a group, whatever — seeking to understand, or seeking to be understood. That’s where I see the fundamental dynamic of where many communications break down, is when both people have very different, conflicting agendas:

      So, a good segue to one of the questions I wanted to ask you, which is: How do you manage… <sure> — and this to me is one of the most top of mind things in this environment today — online, virtual, in person — how do you manage tricky communications? Just at a very high level like, you know you’ve done sessions with me and some of the team on how to manage like at a live event — if you have someone on your panel who’s kind of going in a different tangent; or, you have a spontaneous questioner who comes up and kind of throws a different vibe into the dynamic. Let’s break all of that down, starting with having an unruly panel, if you’re running a discussion, live event, moderating a room… whatever.

      Matt: Sure. So, in all those tricky situations, again, pre-work matters; anything you can do to set yourself up for success: Talking to people in advance, so you set their expectations; giving some ground rules for what you expect.

      If it gets unruly, your biggest friend is paraphrasing.

      The power of paraphrasing

      I really think the ability to paraphrase is THE most essential tool a facilitator needs to have in his or her back pocket. Let me explain first what I mean by paraphrasing then give you some examples of how to use it:

      So, when I’m speaking about paraphrasing, I’m talking about listening to hear what IS the bottom line — the critical gist of what somebody is saying. And this requires a very different type of listening; most of the time when we listen, we’re just listening to get a vague idea of what someone’s saying, and then we begin formulating our response or rehearsing it. But when you’re listening to paraphrase, you’re really trying to figure out, what’s the bottom line.

      And here’s how paraphrasing can really help you: If somebody is going off on a tangent, or if somebody is just bloviating, or they’re trying to figure out what it is they want to contribute — extract something of value (to you or to the conversation that you’re trying to facilitate), highlight it, and then, link or bridge to a different topic.

      So, imagine that you’re about to take us further on a tangent, I can simply say, “Hey, that point you just made about X, that’s really important. And in fact, it ties nicely to…” — and all of a sudden, I’ve taken control back, I’ve validated that you said something useful, and I’ve moved on.

      Sonal: You’re in control.

      Matt: Yeah, it gives you the opportunity to reassert your control in the politest way possible. Because the reality is this: If you’re charged with being the moderator/ the facilitator/ the leader of the interaction, and somebody goes on a tangent, or somebody gets aggressive, or starts really rambling, people are going to look to you to manage that situation. And every moment that you’re not managing it, your credibility is at risk. So you need to step in, but you need to do so politely. And I think paraphrasing — highlighting something somebody said, questioning it in a polite way, whatever that is — is your wedge to get YOU back in control, and then you move it to somewhere else.

      That’s why paraphrasing is often partnered with bridging and linking to the next topic or theme.

      Sonal: It reminds me so much of a podcast host — the #1 thing I think of is that they are a shepherd for the audience. <mhm!> And their job is to do precisely that, the bridging — the signposting is what I call it: what’s happening, stitching things together — and you have to do that a lot in real time.

      So, now tell me more about the bridging and linking!

      Bridging and linking

      Matt: Yeah, so if you have solid themes that you are driving towards — and these are either ones you’ve created yourself or co-created with the other participants — those are the cornerstones or the anchors to which you bridge or link back to. So, if we’re really trying to drive a decision on a particular feature or product, as I am facilitating the interaction, as different points come up, I will always come back to that and say, “how is that”; or, or either ask how it is; or show, and demonstrate, how it is linked to the theme that we are striving towards.

      So it means in advance, you have some guideposts of where you’re going — those are the themes that you’re driving towards — and then you bridge and link back to them. And you can bridge and link back through questioning, “How does that link to our goal”; you can do it directly by saying, “That links to our goal in these ways”; or you can ask somebody else, you could say “Okay Sonal, now how do you think that helps us achieve the goal that we’re striving for?”

      All of those are techniques for bridging and linking back to the central ideas.

      Sonal: You know it’s a lot like a host at a cocktail party, where people are kind of meeting each other for the first time, and you’re like “Oh, you know, Matt, you just mentioned this, well it turns out that so and so is also really into this, and you guys have that in common.” And while that’s more in the sense of get-to-know each other, <mhm> this is exactly the same thing but in the sense of get to know this idea and let me help you kind of connect all these dots.

      Matt: Right and the key word you said there is “connect” — and that’s really what a good facilitator and moderator does; it’s all about connecting. And connecting is just another word for bridging and linking — that’s really the task.

      And it’s a mindset — you have to go into the situation thinking that way — and that’s why I like your host analogy. You know for many of us when we host a party, we have to get into that role and say, I’m a host, it’s my job to make sure everybody’s talking and enjoying themselves and connecting.

      Same too, with a moderator: Many of us go into our role as facilitator or moderator with that contributor’s mindset. And that’s very different than when you are actually in the role of moderating. So that linking/ bridging/ connecting matters, a lot.

      Sonal: It’s so funny because in the early days of moderating on the podcast, I often struggled with, I shouldn’t speak up; I’m here to only set up my host. And then I had all these people (fans, others) messaging me like, speak up more, we wanna hear more from you… and I realized like, oh my god, the orientation point is the voice — they’re the GPS for the episode, the themes that cut across things — and the connecting is key, because in audio in particular, the intimacy you have is SO exquisite. And this is really relevant to communities: like, let’s say you have a club; or, a group of people in the workplace, a team, a department, a meeting, a project — that idea of connecting, I agree, is critical.

      It’s about the thing that the listener wants, the audience wants, that’s top of mind, making it about what you said about why is this relevant to YOU? That is another great orienting technique. Because one of my biggest pet peeves when I go into a conversation, especially in podcasts (or a newsletter blurb, or any kind of editorial product) is not knowing why does anyone care? <Right> Like, that is the first thing that I want to know out the door. Period.

      Matt: I love the analogy of GPS. And, I think that’s a great way to look at it, is: you have a destination; your job is to get there; there are multiple paths to get you there — as a moderator you have to decide, do we take the most direct route, are we going to take some more scenic routes to get there; but you’re really driving towards that goal.

      And I have to say as a listener to your podcast, Sonal, I love when you contribute, and I think there is a role for the moderator and facilitator to share his or her points of view. But you do so in a very… thoughtful way, so it doesn’t just become about your point of view and your direction. And that’s a skill. It’s a skill to learn when and how much to contribute.

      Sonal: It is not easy, and it’s something that I also constantly learn and evolve…

      But just also — because all the listeners of the show know I can’t resist a damn good analogy! — if you take the human GPS analogy even further, and you’re saying you have to know are you taking the scenic route or this route? — in much the same way, when someone’s in the car seat with you giving you directions, you wanna kind of know the map and the terrain ahead of time. Like “by the way in three streets, we’re gonna turn right”. <right> Because you don’t want to suddenly turn right, right? <correct> And, similarly, you want to know if there’s like a lake that you don’t want to drive into by accident <Matt chuckles> like, hey we may want to avoid that traffic jam. So, as a moderator, you’re kind of rerouting around people are going too long on this thing; or, oh man that’s like a- I don’t want to jump into this lake, like, that’s going to tank this conversation. Let me redirect this. So I totally love that analogy, taking it even a step further.

      Matt: Yes, it works really well. For sure.

      Sonal: One of the tactics — you talked about always having the bottom line in mind, as a way to kind of help with the paraphrasing, the bridging, and the linking (it is both the way to summarize the paraphrase, as well as a way to then signal that you’re about to take a turn) — I have to give you credit, because I just realized (I don’t even know if I remember this, but I think) one of my signature lines on one of our other shows, 16 Minutes, which is our news analysis show, <mhm… yeah> I end every episode with “bottom-line it for me”. And I just remembered in this conversation, like oh my god, I think I got that from you, when you were helping me prep for a live panel years ago.

      Matt: It’s definitely a mantra of mine. But you deploy it expertly, so I’m not going to take any credit.

      Sonal: Well, you deserve the credit!

      Preparation and ground rules

      So on the note of prep, one of the only ways to do a lot of this stuff is to do it in real time, frankly. And if you’re live, like a live community room or a live town hall, or anything else. So… tell me a bit more about what goes into that prep, a little bit more concretely? Is it a script? Is it just knowing your guests really well? Is it a prep call? <Matt chuckles> Like, how do you kind of thread that needle?

      Matt: So, to me, it starts first and foremost, by getting an understanding of what it is that I need to accomplish. Is it really about collaboration? Is it decision making? Is it just getting people to know each other? And from that, it’s really important to then think about the audience. And you have to do reconnaissance, reflection, and research. So it might be looking at people’s social media profiles and postings; it might be talking to people who have interacted with these folks. Or, just talk to the folks themselves — and get a sense of what’s important to them, what their attitudes are, etc. That’s part of the pre-work that you need to do just to understand who’s going to be in the space and part of the communication.

      Next, you have to think about, again, the goal; what is it I’m trying to achieve? Now that I know the people, and where they’re coming from, and the purpose I have, I can then craft the goal: know/ feel/ do > information, emotion, and action. A lot of us are really good at focusing on the information: here’s what I want us to be talking about. And, we’re also pretty good at saying, okay we’re driving towards this kind of action.

      We don’t often think about the feeling, the tone — <yes!> what tone do I want the interaction to have? Maya Angelou is famous for saying I might not remember what you said, but I’ll remember the feeling. So, you need to think about that up front.

      Sonal: I am so glad Matt, that you talked about not just the know but the feel. That to me is the thing that I care about the most as a moderator. And I don’t mean that in only a mushy-gushy way like “Oh I want people to feel good.” But I want people to come out of a conversation feeling smarter, and feeling empowered, or more knowledgeable, or that anything is possible, or that they can find a way that’s relevant to them. And also that I’m their advocate, because I genuinely believe I am.

      I think for me — there’s no like systematic technique or at least one that I’m aware of — is trying to find kind of the person’s guiding light. <mhm> Like, what is the thing that drives them or makes them passionate about what they do? <yeah> And then how do you really draw that out? And we never talk about that actually, overtly.

      Matt: Right. The way we have to actually do it often is much more subtle and nuanced.

      If you feel that the thing that is most important is to convey those feelings, as part of the interactions you’re facilitating, then the question and challenge for you becomes, what do you do in preparation of the participants — during the interaction and even after — to really bring those emotions, those feelings to life? You know it’s so much easier to think about the knowing piece — Here are the bullet points I need to get across, <exactly> here are the questions I need to ask. But what is it that you can do to really call out or invoke those feelings that you want? And it could be simple things: Non-verbally acknowledging what somebody said; it could be thanking somebody and expressing gratitude.

      You then need to stockpile questions. And these are questions that you can use to ask the participants, to get them communicating, to move it in the direction you want. These can be what I call “back-pocket” questions — emergency questions that YOU deploy, if silence comes in — you know, you can throw out a question that says “something I’ve been wondering about,” or “think about how this applies to” these situations. So, having questions you can ask others and having questions you can use to get the conversation moving: really important.

      Sonal: You mentioned “stockpiling”, I want to probe on that one a little bit, because, frankly, I am actually not a big believer in… So okay, Margit calls bullshit on me on this, which I actually really love. But where I’m like, “I don’t believe in prep.” And she’s like, “What are you talking about? Your whole lifetime is prep. Like, you read all the time. You absorb things all the time. Blah, blah, blah.” <right> Which, okay, that’s fair.

      By prepping, I mean like having a script in front of me <mhm> because I want things to be very organic and very free flowing: I’m going on the same journey as my listeners. However: I had one person a few years ago say, “Oh I love being a naive questioner.” And I’m like, Oh, no no no no; you’re not a naive questioner, because that is also bad; <right> like, don’t make that mistake.

      On the flip side, other people go so far with the stockpiling, as you described, <yeah> that they go to the point where they almost lose their way if things don’t kind of stick perfectly, <mhm> and it feels very constrained and scripted. What would your advice be on how to thread that one?

      Matt: So, you’re highlighting a really important point: You want to feel as if you have a direction, and tools to help you get to where you’re going; but you don’t want to have it SO scripted, and SO structured, that free-flowing, spontaneity is stripped from it.

      So, everybody needs to find their level of comfort. People who might be newer to a topic, newer to a language: Doing a little extra prep and scripting could help them. For people who are more comfortable, more extroverted, it might be better to have less of those guidewires. But the point is, you would never go into a situation totally unprepared. You have ideas, themes; you have some boundaries.

      I love this research (it came out of the U.K.), what they did is they took children and they brought them to an empty field and they said “go play.” And the children played. And the researchers evaluated how playful the play was, how creative the play was, how much time was spent playing versus planning. And then they brought a similar group of kids to a similar field, but the difference was, in the second field there was a play structure. And they said go play. And they rated the same things (amount of play, quality of play, creativity) — and it turned out, the play with the play structure was much more creative, much more engaging, more time spent playing. I like that as an analogy for planning interactions: Having some structure, some tools, some idea of content, direction, etc., can really, really help you focus on what you’re trying to do. If it’s TOO open, if it’s TOO spontaneous, you can get lost in that spontaneity.

      So, finding the right balance is hard (each person is different), but using that as a guide — knowing you have to have some structure, some tools, some things in that stockpile — can really help.

      Sonal: I found that research so fascinating, because I was in the world of early education and developmental psychology as you know back in the day <right> — and one of the one of the concepts (the phrase in the education world, this constructivism idea) — was “scaffolding” versus structure. And the idea is that it’s like the bones — it’s not like a full- built structure, but the scaffolding that sets something up, but it’s not fully filled in, and it’s also not like fully free-for-all — so, that’s an idea that applies there.

      And then two, the other thing is the importance of ground rules. Because one of the things that you learn with early childhood education and any kind of play, is all the kids going into it know the ground rules: Like, you cannot hit, you cannot fight, you cannot pull so-and-so’s hair, or you know, wear sun block <chuckles>; whatever the rule is! <chuckles> <Matt: right> So, I’d love to hear you tell me more about how you think about the ground rules to make these goals, and intentions, and scaffolding more explicit — versus only in the moderator’s head — to the audience and the panels.

      Matt: So first and foremost, there’re two different types of types of ground rules: There are behavioral ground rules, that’s what you do, how you act; and then there are content- specific ground rules, <mmm!> what’s acceptable to say and what’s not. Just creating those two categories can be helpful for people.

      Now, to the question of how do you share them: So first and foremost, you can take time to collaborate together to create them. So you can start by saying, hey let’s figure out how we want to best interact. By virtue of co-creating them, that’s how you’re disseminating the information. If you want to do them in advance, come in with them… then, you can put them in the invite to the meeting, or in some communication that happens in advance, and then just remind people of them when you start.

      What you want to avoid with any rules that you set up is getting bogged down in the rules. If you have ever watched young children (and I know you have experience with this), young children interacting, they spend a tremendous amount of time just dealing with the rules — so much so, that they don’t actually get to playing whatever it is they’re trying to play. And adults can do the same thing. <yeah> So, it’s make them explicit; maybe create them with others; and then just get moving on, from them.

      Sonal: One other question about knowing the audience’s intent in a live event, where you may not have the ability to know — like, for example, parasocial versus social interactions, where you’re interacting with strangers, often, in a group of people — so how do you then think of aligning the goals and knowing your audience when you have groups of strangers interacting in the same room? This is the case that’s common when you go to a conference and there might be unknown people who can just come and join the Q&A section; you don’t have registration, it’s an open event or, it could be in online audio social places like Clubhouse… it plays out in many different ways.

      Matt: Wouldn’t it be great just to be psychic and be able to know that stuff? That would be fantastic! So, I mean look for contextual clues: what’s the title of the event; what’s the motivation for people to be there — and that can often give you cues as to what’s important to people.

      The other way is just to inquire, ask questions; observe what people seem to be saying and how they’re saying it — gives you insight into what’s important for them. But again, that means your approach is different than coming in as, I’m a contributor and I’m gonna share what I have <yes> on my mind. Versus, I need to understand what’s going on and taking that time just to reflect and look around and see what others are doing can be very helpful to figuring it out.

      And then, being comfortable adjusting on the fly <yes!> — I can’t tell you the number of interactions I have gone into where I thought we were going one way with this group of people, and it turned out to be different. And you just have to be flexible and say okay, that’s what this is going to be about, or that’s how we’re going to make this conversation move forward. And, you know improvisation — the notion of “yes and” — take what you’ve got and move it forward, rather than coming in and say this is what “this conversation is going to be about”.

      And certainly there’re times that you have to drive the conversation to a particular point; but a lot of the time, we can just see what happens organically and move with it, within the structure and confines of what we’re talking about.

      Sonal: This goes to me to how I think about prep docs, ‘cuz while I don’t stockpile questions in advance, I do have like a quick-list of topics that I want to make sure to hit. <sure> And it’s really helpful, because I know the three that I absolutely want to hit no matter what, but then I also have like a couple others that may come up, that I can go into and pull (or double-click on so to speak) — if it’s more interesting. And if it’s not so interesting, then you quickly can move into something else, because, you kind of want to always think about what’s maximally interesting to keep people engaged.

      So, the way I structure my prep docs: I make ‘em modular chunks, so that I can go out of order very easily. And I know this is a piece of advice that you probably also have given. But for me, that’s like the #1 thing is, I have an arc in mind <mhm> but I keep it very modular chunks so that I can quickly rearrange it on the fly if necessary; I’m not wedded to that.

      Secondly, like a quick topic, I might have like a one word or two words for like a probe <mhm> — like angle, or twist, or nuance — because that’s kind of the thing that makes it more differentiated from like the same way of having that conversation.

      So, I have like a particular template that I’ve made up over years of doing a lot of these, that works very well for me in this vein.

      Matt: I would love to see the template… I absolutely agree that “chunking” or being modular is really important. And, having just key topics that you want to address can work very well for many people.

      The only thing I would add to that is try to have some prioritization among those, because if time gets crunched, or, if some topic heats up and takes you in a different direction — know the prioritization, so you can adjust. So on the fly, you’re not having to make those decisions, you’ve already thought about this is the most important, this is second and third most important.

      Sonal: Oh you’re absolutely right. And sometimes I, in my template, conflate arc-order with priority. But in fact, sometimes the last thing is the most important thing to get across. And so having that prioritization is really critical.

      I will also add that I don’t map it out like time-wise, but I put percentages next to each modular chunk in order to kind of figure out the weighting of it: So, I want 50% of the conversation to be about this; and then like 20%, like takeaways — that’s not quite the same as priority, but it does tell you how much you want to get across.

      Matt: I am smiling as you are speaking. Not only do I like that idea, but I, like Margit, am gonna call bullshit that you don’t plan and prepare. <Sonal laughs out loud> I mean everything you have just described is planning and preparing to an extent that most people don’t — even if it doesn’t feel that way… so! <chuckles>

      Sonal: Okay but to be very clear, I only do that for live events. I do NOT do that for podcasts; I’ll tell you what I do for podcasts: I quickly, at the very beginning, spend five minutes — and we have obviously the general theme because of the guests, and the lineup, and the angle — so what happens is, when I get people together, and it’s usually multiple people, we quickly talk about — and I say very clearly, I want topics, I don’t want you to tell me what you’re gonna say <mhm> —

      And in fact, one of my fundamental rules of live events, is I do not believe in putting people in the same green room beforehand. Because speakers reference something — they always do this, like “oh yeah, we were talking about this in the green room” — and the audience is left feeling like they were cheated out of the idea. And so I don’t want any rehearsal. I actually cut people off when we do this, in the first five minutes, where I’m like “No no no no no — save that for the actual discussion. I don’t want you to tell me what you’re going to say. I just want the topics.” Because nothing ever sounds as good as the first time someone says it raw, and real-ly.

      Matt: I agree. And as a facilitator and moderator, your job is to bring out that fresh conversation. And if people do talk about private, or previous conversations, you have to call it, and you have to bring it forward to make it relevant to everybody.

      One of one of the best mindsets or frames that a moderator/facilitator can have is that YOU are the voice of the audience. <yes!> So if there’s something that is inside baseball, if there’s some insider information, you have to call it, you have to pull it out so others can participate.

      And there are things you can do that are very simple linguistically: You can say, “as we’re curious”, or “as you know”, or “as many of us are interested” — using that inclusive language brings the audience IN. Not only does it help the audience feel like they’re part of that conversation, but it reminds the others — the panelists, the people that you’re helping facilitate — that there’s an audience they need to be talking to, it’s not just themselves.

      Sonal: It’s not talking to each other — I love this. So this goes back to the host being a shepherd.

      But actually, you talk about the linguistic aspects — this is one of my favorite technique that I’ve specifically learned from you (in some of the live event preparation) which is: How to change the exact same question, but in a way that it’s very much phrased as advocating on behalf of the audience. And you went so far as to even show me physical, nonverbal things that I can do to bring the audience along, where, I literally open up my hand like “listen, I think everyone in this room” — kind of hug the room in <right!> — “wants to know like, what do you mean by that”? <right> That was SO useful.

      On virtual communication

      Matt: Yeah, it’s not just verbal stuff that you can do using words — using inclusive language, using analogies that everybody relates to; ALL of that’s a way to do that verbally — but nonverbals matter a lot. Now the fact that we’re virtual, it’s harder.

      The equivalent to what you mentioned — where you actually open up your body and angle it towards the audience, as you say “as many of us in this room are wondering,” before you turn to the person and ask the question <mhm> — the way we have to do that virtually is you have to look at the camera. And it’s SO tempting to look at notes or to look at the faces on the screen, but you need to look at the *camera* so that people feel like you’re connecting TO them, talking TO them, and including them. And that’s hard.

      Sonal: I am so glad you brought up the online/ remote environment. Because a) I don’t think this is an important skill just for the duration of the pandemic — let’s face it, a lot of knowledge work in particular is gonna be remote-first — we’ve definitely shifted the baseline on this. But secondly, I don’t believe we’ve seen the first big wave of companies that are all built in an all remote-native way — culturally, interaction-wise, etc. — it’ll be really interesting to see a lot of the learnings that come out of that, because we are in an unprecedented age of online communication and collaboration.

      So, can we really dig deep into both nonverbal and in-person, and then let’s go into nonverbal and the differences online. Like, how does one optimize techniques, like, I open up my arms in a room — but in Slack, nobody even sees my arms. How do you… think about all that.

      Matt: So there are three major components to nonverbal presence: There is *the visual, *the vocal, and *the verbal. And these play out differently depending on the channel through which you’re communicating (in person, online, et cetera).

      The importance of visuals

      So visually is what people see of you; it’s how you hold your body. We have to make sure that we come across as confident and composed. So we want to be big (that is, not hunched or crouched); we want to be balanced (head straight, shoulders square); and we want to be still.

      Now everybody has to find what’s comfortable to them; you know I always give the analogy, we could ask every one of your listeners to show how they swing a baseball bat, a tennis racket, a golf club — how they look for each person is going to be slightly different because of their build, their experience, their injuries. And that’s what we strive for in our nonverbal presence: You follow some foundational principles, and then you adapt them to who you are and your experience.

      So, visual is what we see. And virtually or in person — big, balanced, and still is what it’s all about.

      Sonal: How do you do big in virtual though?

      Matt: Ahh, great question. So when you’re in that little box — whatever the tool is you’re using, we’re all in our little Hollywood Squares, Brady-Bunch boxes — you want to pull your scapula, your shoulder blades down, away from your neck. And in so doing, it broadens out your shoulders. So you look bigger, and you sit straighter. It also will tense the muscles in your neck so your head doesn’t tilt; head tilting in a virtual environment might compromise your credibility and confidence (or at least appearance of that).

      So, when you’re in the box: Pull those shoulder blades down, broaden the shoulders, hold your head straight; really important.

      The other thing that’s important is gesturing: When I’m up in front of people, I want my gestures to be broad; I want them to go beyond my shoulders. Now when I’m virtual in the box, if I were to do that, you’d never see my hands.

      Sonal: It looks weird too, when people even wave goodbye.

      Matt: Yeah, no it is weird! But gesturing is important. Gesturing helps your audience, it also helps you.

      So bringing your hands up higher, putting them about your shoulder level — so if I were to see you in person doing this, you would look like a caricature, a puppet — but online, in a virtual meeting, it actually looks okay to have your hands up. And then again, broader than your shoulder — we want to avoid any gestures that are in front of our chest for too long, because it makes you look tight and nervous. <right!>

      On vocals

      So that’s the visual part. The vocal part is varying your voice. You know this so well; I mean, with podcasting as a medium — if. I. talked. like. this. for even. just a few seconds, folks are gonna tune out. <Yeah, Ferris Bueller effect! chuckles> Exactly! Our brains are wired to look for and seek out novelty and change, anything that stays the same, we habituate to very quickly. So you need to make sure that your voice has variation in it. <yes> And a great way to bring that variation is to use emotive words, adjectives and adverbs. So I would never say “I’m really excited to be here Sonal.” I would say, “I’m really excited to be here!” <Sonal laughs> So really in the “excited”, invoke that emotion.

      So you want to have variation. And really, what it comes down to in person or virtually, you have to work on your breath; your voice is a wind instrument. And if you don’t have vocal stamina, you’re gonna be in trouble: Your voice is gonna trail off, you’re gonna start speaking fast. So I encourage everybody, before you have a big event — I don’t care if it’s a presentation, meeting contribution — you should be building vocal stamina. And the best way I know to do that is reading out loud.

      So if I know next week I’m doing a 30-minute whatever, I’m reading out loud the week before 5-10 minutes each day to build stamina. I equate it to, if you want to run a run a marathon, you don’t start at that distance; you start by doing gradually more and more mileage. The same thing has to be true with your voice. That way you can support your voice — and therefore your ideas — as you speak.

      So, breath control is critical.

      Sonal: I’m definitely gonna come back to that one, because I have a lot of thoughts on that one!

      So: so far, we covered the visual and the vocal. So let’s do the third one.

      Verbal patterns

      Matt: So let me talk about the verbal. So clearly, the words you say are important. What I really like to highlight are the words that get in the way, what I call the “verbal graffiti” — so it’s the ums, the uhs, the likes, I means — my favorite, “honestly”, that one bothers me so much, because it implies everything else you said prior was dishonest — we use those fillers. And, it is really hard to get rid of them. The best thing you can do is just try to build your own awareness. And based on that, then, eventually over time, they will decrease.

      The other part of verbal that I want to add is hedging language; this stuff, it is rampant: kind of, sort of, I think — that language undercuts your credibility. If I were to say, “Sonal, I kind of think we should do this” versus “we should do this”, it just sounds very different. Now there are times, if I’m leading a meeting, and I’m the head honcho and I want to avoid people just doing what I say because I’m the big boss, then I might say “I kind of think we should do this” — because that invites them to share their opinion. But when you are running a panel, when you’re giving a presentation, and you say “kind of” and “sort of” and “I think” all over the place, you are reducing your credibility.

      Sonal: Oh my. So first of all, I love the framework, super helpful; because you’re actually reminding anyone, in any speaking engagement — you are visual, vocal, and verbal — it feels like it’s obvious, but it’s really not; because when you go into any session, it’s so important to tease them apart, so you keep all three in balance.

      So let’s start with the first one, which is visual. One question I wanted to just check in with you about is, when it comes to Zoom meetings is like visual fatigue — <mhm> no one looks at each other in a meeting where you’re literally looking eye to eye the entire hour — and so there’s a visual exhaustion that happens. And then secondly, it’s very hard to tell where to look. So can you give me a few more specifics about where the eyes should go and land? Because one of the techniques that you’ve taught me in live events is to land your eyes. <mhm> But, how do you even do that when you don’t know; it’s like a black hole!

      Matt: Eye contact virtually is really challenging. It’s challenging because, where the camera is and where you want to look are two different places: So we want to look at people’s images, if people are showing their video; and that’s usually below the camera. And what it looks like to the audience, if you’re actually looking at the pictures, is that you’re talking to their feet. And we know that that’s rude in person, and part of us says, hey look at me. And we attribute a whole bunch of negative thoughts to people who don’t look us in the eye: They’re nervous, they’re not prepared, <totally> they’re lying. So you really do need to train yourself to look at the camera.

      So, a couple things you can do to help: One, some of the virtual tools allow you to physically move people’s images; so you can actually move the images under or closer to where the camera is. Other times (what I recommend people do) is take a picture of people you know, or maybe even a pet you own, and put it right behind the camera — we as humans are wired to look at living things, so put a picture right behind it — and that will help you remember to look and connect to it.

      The other thing that’s really tricky here Sonal is, we are not used to seeing ourselves when we speak. There’s research that shows it activates areas of our brain regarding self-awareness, that we typically don’t have active when we’re communicating <right!> — and it drains cognitive resources. So, some of these tools actually allow you to mute your own image; I know somebody who takes a post-it, sticks it right over her image.

      But just know that seeing yourself speak is hard.

      Sonal: You’re absolutely right. I use it unfortunately, as a mirror, <mhm> <Sonal chuckles> where I’m constantly checking myself, like wait my hair’s out of place — and the other thing is when you go to a live event, you know they have confidence monitors; and in this case, it’s like the opposite of a confidence monitor: it’s like an un-confidence monitor because it’s really distracting.

      So, I love that tip of putting a post-it. And I also forgot that some tools allow you to turn that view off — but it IS incredibly different — because when you’re on stage, you’re not that close up. It’s a new level of intimacy and I actually think we’re going to see some new behaviors come out of it, and maybe with new technologies, even better <mhm> — but it is not easy, for sure.

      Matt: Yeah, no, it’s not easy. And I think as we do more and more of this, we will get more used to it.

      Sonal: Yeah, I agree. Okay, so then that’s for the visual. So now on vocal — the second part of the framework <mhm> — we talked about varying cadence. And god, as an podcast editor, what’s really fascinating to me is how most of the time, people are off in their cadence, like it’s misaligned. So for instance, the moment they should be slowing down, they’re speeding up; and the moment they should be speeding up, they’re taking too long to get it out. And I do this too, for the record. But I noticed when I interviewed Guy Raz — who’s obviously a very seasoned radio <mhm> and voice personality — the edit was kind of easier than other edits, because every sentence he gave was so clean. <mhm!>

      And I was like oh my god, this is a technique of a really trained voice personality, essentially — and that’s a new type that’s emerging in this modern era of audio: “voice personas” — where, the better you are at varying your cadence — Like he would do things, like he’d slow down… when… it’s about to get really intimate… and… special. <yeah…> And that immediately, instantly makes you viscerally respond — both as the guest and the audience — so it’s really fascinating how that plays a role.

      I also love that you talked about using an adjective, like something that makes it emotive. Because you’re right, you can’t say the word excited, like “I am so excited” you know <laughs>.

      Matt: You have to work at it.

      Sonal: Right, you have to work harder to NOT do that. I will also say though that this goes back to your earlier point about the feeling, and the tone of the room, and setting up that how you want people to feel — because the better you are a master of that, then the better you can actually control that. <Matt: Absolutely>

      Breath control and anxiety

      And then the final thing is on the breath now. And we’ll come back to this on the anxiety part but it is very tied, as you know Matt, to anxiety. And it’s really hard when you get anxious about public speaking to manage your breath. I often feel, when I go on stage (for live events, this is, because that’s what I’ve worked with you on) where, I feel like I can’t get my breath. Like, I’m going to have a panic attack or something.

      So, can you say more about the breath? I mean, you gave some for proactive planning, but can you give us some in-situ, like reactive things to do to control your breath?

      Matt: Certainly. And you are not alone. Being nervous and having it affect your breath happens a lot.

      So one of the cool things about being virtual, is you can mute yourself. Taking deep breaths to help calm yourself down has been known for millennia. And, I can just mute myself, take a deep breath; nobody’s the wiser. Much harder to do in person — so there are some advantages that the virtual world brings us.

      If you find that in the midst of communicating, your breath is getting away from you — because you’re nervous, or because you’re getting excited — we human beings sync up three things: the rate with which our eyes move, the rate with which we speak, and, how quickly we gesture. It is very hard to change your eye movements; it’s reasonably hard to change your breathing; it is pretty easy to change how fast you gesture. So, if you find yourself breathing quickly and out of breath, slow down your gestures, make them a little broader — it will slow down your breathing. And that’s something everybody can do, in the moment, that can help a lot.

      So taking a deep breath before; working on your vocal stamina (way in advance of ever doing a communication event); and monitoring and managing your gesture rate can all help you breathe more evenly and less rapidly.

      Sonal: I have one more from you, and one of mine.

      One from you is — and this goes back to your earlier point of having an emergency question — which is how to have that in your back pocket, so that if I do find myself– not only is it useful if you kind of lose your train of thought (which does happen a lot in real time), but, it’s really great when you’re feeling like that anxiety coming on. Because you can get that question out, and then it lets you catch your breath while people answer.

      And the other one that I — this is going to sound so funny — but it’s just taking a sip of water. It’s huge; because it’s another way that you can kind of slow down and catch your breath. I always tell audio platforms that one of my favorite features that I want everyone to build is a “drink water” button, and everyone kind of chuckles but I’m like no I’m serious, <Matt chuckles> this is what I really think is important.

      Matt: Absolutely. Taking a breath, actually physically just moving — you don’t have to speak as you move, and you can take a breath as you step — it’s a great way, especially if it’s a transition point.

      Sonal: So we covered the nuances that you outlined in the framework of visual, vocal… now let’s go into verbal. One thing I wanted to talk about here, with what you brought up, is, the verbal tics: So first of all, I agree with you; they are very weakening words. But: I do not believe in eliminating every single tic — I actually think that’s very bad practice, because we’re wired to hear people sound real and raw. And as you know, everyone has them. <right> My rule of thumb that I tell the audio editors is, try to remove as many tics as possible that are disruptive to the listener’s experience <mhm> — so if it’s like a “That’s right. That’s right. That’s right. That’s right.” — it’s almost like annoying to get the point across, then cut those. But otherwise, keep ‘em, so it’s not like robotic either, you know.

      However… of course, I have a lot of vanity tics. And so I tried to get rid of them. Early days of podcasting, I was always behind the scenes; so I hated hearing my own voice, all of that. I always noticed only the tics — I like, I like, I like. Got it, got it, got it. Right, right, right — I have a million, and they’re so freaking annoying <mhm>. So I’d like systematically try to work on not saying them. And as you note, one of the ways to do that is to record yourself and hear yourself.

      Guess what happened?

      Matt: What?

      Sonal: Another one popped in its place! <Sonal chuckles> <Matt: uh-oh!> So I got rid of “I like” and the next one was, “got it”. I got rid of that one, and then “right” came up, and then something else came up, like, “uh-huh, uh-huh”. And I think they serve some neurological purpose — I don’t know if you have a thought on this, but I think it’s impossible to get rid of tics.

      Matt: Well I know it’s not impossible, because I’ve done it and I have helped other people do it — <Aww, dammit!> <Sonal laughs> but you’re right, they don’t ever go away completely. They don’t go away completely, but you can reduce their frequency.

      I believe that they are remnants of our thinking, and in-the-moment feeling like we need to be saying something because we are in front of people; <Yes> we’re filling the space. And that’s why they’re called filler — <filler words>

      So, there is a trick, there is a trick — it is hard — but there is a trick where it is a breathing issue. So, speaking is an exit-only event: You can only speak when you’re pushing air out, not when you’re taking air in. So if you happen to know that you say “got it” or “right” at the end of all your sentences or phrases, if you can train yourself to be completely out of breath when you are done speaking that phrase, you must inhale before you can say your next phrase. <ahh!> Which precludes you from saying anything such as right, um, got it.

      Now that’s hard… <mhm> But as you were referring to earlier with Guy Raz, you can train yourself to really end and finish your sentences. <mhm!> And then you start another one. And by training yourself to land a phrase — to finish a phrase completely out of breath (now I’m not saying get <quiet at the end>, I’m just saying finish a phrase) — you then have to inhale, builds a pause (pauses are good), and doesn’t allow you to fill it with anything.

      Sonal: I am going to try that.

      People complain all the time about how we are all very fast talkers. <yes> And it is true, I talk the way I think, and maybe I could slow down on that. <chuckles>

      Matt: Well, it’s interesting — ‘cause I don’t find you a fast talker — but what I find is sometimes <mhm> you won’t pause as long as you could. I speak very quickly too, but if I pause… people can catch up. The problem is, the listeners get fatigued <yes> because there’s no rest.

      Sonal: I find that too; I also notice that and it drives me a little nuts that I do that, some of my speakers do that.

      You know what is funny? — people don’t know this — a lot of people think we cut all the breaths out of our podcasts; it’s actually the opposite.

      Matt: Oh really?

      Sonal: Many times in an edit, we are often going in and adding breaths, because, I needed to slow it down to give the listener a split second to take it in. Exactly to your point. And I don’t do it myself.

      The other thing is just I want to make a note, with the filler words: Sometimes I think it has to do with representation, sometimes I think it has to do with just societally; in fact, one of the edits I make often, for a lot of my expert guests, is NOT having them say an acknowledging statement at the beginning, “Well, you know, Tom, I agree with you, Jim. But here’s what I think.” And I just go right to the “I think”, which is such an important thing.

      Matt: I agree with everything that you’ve said. And it’s — the kind ofs, sort ofs, I thinks creeps into everybody’s language — I hear it more and more across <yeah> everybody I work with.

      Sonal: Yep. I hear this across very established, privileged, powerful people <yes> — all the time, everybody has them; so it’s not at all disproportionate in that sense. (I do think it’s dangerous when we judge the speech of people, like no vocal fry, or women shouldn’t do this, or uptalk and whatnot — which you’re not doing at all; it’s really about how to make the authority come across.)

      New audio platforms

      So one last thing on the visual, vocal, and verbal — there’s been an emergence of social audio and new forms of audio-interaction platforms, like Clubhouse, and you know there’s a whole wave of other types of tools for different interactions; gaming contexts, others. And I, I have to tell you, it’s completely changed how I think about communication — that framework you outlined, if you’re in a voice-only medium, you almost have to caricature-like, exaggerate some of the things that we’re talking about to make up for the lack of visual.

      Matt: It’s really interesting you bring that up; that is going on concurrently with people wearing masks, where we also have to exaggerate nonverbal behavior <oh yeah… totally!> to communicate information. So, we are in a position now where nonverbal presence — both in vocalics, what you do with your voice and what you do with your face, etc., — are really being highlighted.

      And for most of our lives, we really haven’t thought about that. For some people, this is exciting and liberating; for other people, it’s really, really challenging. But, you’re right, <yeah> we are having to focus on… emphasizing things very consciously… to get our points across because something in our situation is different: We’re covered up, we don’t have the visual cues.

      Sonal: The other thing that’s happening in a lot of these new interaction paradigms is, it’s often more social-first, by default, than content-first, necessarily — even though it is about content and interaction. And so one of the things that I’m kind of learning is, how to navigate that. And so the question I have for you along these lines, is — we’ve talked already about how to deal with like navigating tricky panelists, navigating tricky audience members — what have if actually want to proactively, offensively engage a tricky conversation, socially, oftentimes with strangers. I’d love to know if you have any thoughts on that, and which best practices may or may not apply.

      Matt: So I find that very intriguing, to actually be an instigator of some tension and conflict. That’s very provocative.

      You know, I am a big fan of using questions to invite engagement, participation, and in this case, perhaps challenges. People can come in with declarative statements that can be seen as, as offensive and really make people defensive. But if you’re really inviting, I think questions are the best way to invite. So for example: When I give people advice on giving feedback, a component of feedback is an invitation to collaborate to fix the problem. And that invitation is best delivered as a question, I believe.

      And for what you’re talking about, using *questions* is a great way to do that rather than come in with some exclamation or declaration.

      Structuring a talk

      Sonal: Great. So the other key thing that I’ve noticed in these kinds of dynamics when you have parasocial and social mixed — you know strangers and familiars — is intent matters. And to me, one of the greatest sources of conflict is when you have two competing intents: One being, I just want empathy; and the other being, I don’t want an echo chamber, I want to hear other competing viewpoints. And so to that point — now I want to ask you about how that plays into concretely, how do you then design the beginning, middle, and end of a session; whether it’s a live event, a room, a panel, a meeting. How do you think about structure in that?

      Matt: So structure is something I spend a lot of time thinking about. And, I think about it from an overarching event structure — so the meeting itself, the panel, the presentation — but also the specific content that gets discussed in that: be it a contribution you’re making, a presentation you’re delivering, or in an interaction you’re facilitating.

      So, at the macro level, it’s all about the arc — this is where we can look to artists, look at playwrights, look at movies — look at how do people weave… that? What do I want the beginning to feel like? What information do I want at the beginning? Where do I want to land this? And then there’s the actual content that gets spoken in the actual interaction. And for that, I can give very concrete examples. <please, yeah>

      So I am a huge fan — a huge fan — of structure. And the structure that I like the most for information is what I call *the what, *so what, *now what structure.

      And, let me explain how it works: It starts by defining what it is you’re talking about — could be your idea, your product, your process. You then talk about why it’s important, that’s the “so what”. And you get to pick the level of relevance here — it could be to the individual you’re talking to, could be a group, could be a company, it could be society in general. <Yes!> And then “the now what” is the next step, what comes next? Maybe it’s signing up for a particular offering; maybe it’s calendaring another meeting; perhaps it’s looking at a demo, or having somebody else come onto the stage.

      But if you can package your information in a way that is clear and concise and connected, then it’s going to be more valuable. And this structure really helps do that. And you can move things around. So if I’m talking to a hesitant or resistant audience, I might move the “so what” first. Start by saying, imagine what it would be like if we could save money, or time, or lives? <Yes!> And people are like yes, I like that. Then you say well, here’s what we need to do: Here’s the what, and here’s the now what that comes after it.

      And it applies not just to information you’re disseminating, it could be feedback you’re giving, it could be emails you’re writing — a structure like what/ so what/ now what can help. So when you put the micro-level structure — the what, so what, now what, into the macro-level structure — where you’re worrying about the flow and the arc, that’s where you get rich, engaging, memorable communication happening.

      Introductions and conclusions

      Sonal: That’s fantastic, Matt. And I love what you said about that you could reorder it based on resistance, because, <yeah> that is exactly how I think about every podcast episode or event is — it is not just about the topic, it’s actually about broadening the potential audience for the topic. And so you can actually bring more people in if you orient things in a broader way — like, hey this conversation seems like it’s about DevOps <right> but it’s really about innovation and all of you care about this, actually. And “the now what”, I think of as how do you know bridge theory to practice, or, make something more concrete — like, you were talking about abstract software system — what do people DO with this information? <right> Or what do people act on? And I think that’s a very, very useful, framework.

      And in fact, it frees you up! Because one of the techniques that very good playwrights (to use your example), use is the technique of “in medias res”, like starting something in the middle of the action <mhm> — you know like the way Star Wars began; <yeah!> it doesn’t begin with like, episode one, it begins with episode four — and in that way, we can actually start the conversation by picking the right place: And the way we orient it, is the what/ so what/ now what!

      Matt: Yeah, and I’ll just make one other comment — I totally agree with the notion of starting with action, starting in the middle — there are a few things I get up on a soapbox for, and I really really want to see changed in people’s communication — I would love for presentations, meetings, and panels to avoid starting with “Hi, my name is; today I’m going to talk about”. The analogy that I use is every action movie starts with Action. And then they put up the Title. And then they put up the Credits. <Exactly!> And I would much prefer that you start with something provocative, intriguing, interesting — and then say who you are and what you’re going to cover. And it gets right to that point you talked about: start in the middle.

      So, not only do you have to think about how you structure the event, and how you structure your content, but think about how you structure the START.

      Sonal: My biggest pet peeve is when people have the guests introduce themselves <mhm>, because a moderator is literally conceding control of how to begin the conversation in the most boring way possible. Even if you tell them, do it in 30 seconds or less, it does not set the tone that you want — in fact, I very strongly believe a moderator needs to do the intro for their guests — you can get that bullet point across in like two words <yeah> instead of wasting like three minutes on it. It’s the worst use of time, to begin any conversation.

      Matt: I absolutely agree.

      Sonal: So on the intros, you said it’s really important to understand your audience — and one of the techniques is to understand their context or cues — what do you make up the technique of polls… especially in a parasocial community where you don’t really know everybody and you want to sort of understand. How does that fit in or not fit in? What do you think about polls and polling your audience?

      Matt: So I think anything that gets your audience interacting is a good thing — rhetorical questions, questions the way they answer — polls are very useful. But polls work in a limited way; you can’t keep polling your audience.

      Two rules for using polls: You have to tell people how to respond. And second, you have to comment on whatever response you get. If you just throw out a question, and people don’t know am I thinking the answer, am I raising my hand, if it’s virtual do I push on a button? — so you have to tell them how to do it; and then comment: Say, oh, that’s what I thought most of you have; or oh, I’m surprised only half of you have. That recognizes the contribution and makes people more likely to feel that it was useful and they’ll do it again.

      Sonal: Great. And then, conclusions! This is one of the techniques I learned from you because I used to be very front-loaded, like only focus on, when live events, on the intro and the middle. And I’d kind of be sloppy at the end like — okay, we’re done — <laughs> I mean, I wasn’t quite that sloppy, but, you know <laughs>

      Matt: Most people think, you know, if I can get the beginning down, then it’ll all follow. But the reality is it doesn’t. Most meetings and presentations end very poorly. In fact, people will just say “Uhh, I guess we’re out of time,” <yeah…> and then they’re done. That’s it.

      Sonal: …Very abrupt and useless. How do you recommend people conclude?

      Matt: Very concisely. I like endings that express gratitude, and then, have a quick wrap up. Quite frankly, if you define a goal up front, then the way you end is simply by stating your goal: “Thank you for your time today. I hope you’re leaving knowing this, feeling this, and likely to do this.” And then you’re done.

      You know as a teacher, I see this all the time. When I signal to my students that we are done or coming close to wrapping up, they are packed up and halfway out the door before I’m done. So that’s why I like ending in a concise and clear way, and being very thoughtful about it in advance about how you want to end.

      Sonal: And I would add one thing, that I’ve learned from editing written text. I don’t like it when conclusions introduce new information. <oh yeah> It’s almost like giving people a teaser that you don’t get to pull that thread. <yeah> It’s okay to allude to something coming, to say we’re going to cover this next time or, <right> stay tuned for the next event on so-and-so date; that’s fine, but I can’t stand it when people bring up a new point in the conclusion.

      Matt: Totally agree. It’s all about concision in the end.

      Managing speaker anxiety

      Sonal: Last question. We’ve threaded through this a little bit throughout the conversation, which is how do people manage anxiety — and that’s of course a psychological question — what would your best tips and advice be (kind of universalities for) how to manage anxiety in both public speaking, written communication, etc.?

      Matt: So I could spend a lot of time talking about this point. I spend a lot of my life helping people become more comfortable and confident speaking; I’ve written a book Speaking Up without Freaking Out on the topic — and it’s something that I think is so critical, because I know we miss valuable input, voices, and ideas because people are just too afraid to share them.

      When it comes to managing anxiety, at the highest level it’s about doing two things: managing symptoms, and managing sources. Symptoms are the things that your body experiences: <mhm> Your hands get shaky? Does your mouth get dry? Do you get sweaty in your brow? And then it’s sources, things that actually exacerbate the anxiety; it’s: Am I worried about trying to get it right? Am I concerned that I might not achieve my goal? Is it that I’m feeling soo intensely evaluated? Those are sources.

      And, with both symptoms and sources, there are things that you can do, that over time, will help you feel more comfortable and confident. It takes work; it’s not a light switch — it’s not like boom, all of a sudden, you’re not nervous. But gradually, you will feel better.

      Sonal: So give us — and I agree it’s a whole longer conversation — but give us a few tips for both symptoms and sources. Some concrete things that people can just do out the door, right away.

      Matt: Sure. So we’ve already talked about deep breathing. Deep breathing will slow down the fight-or-flight autonomic nervous system response that happens.

      People who get shaky — that’s the adrenaline coursing through their bodies — doing big broad gestures when you begin a presentation invokes muscles, big muscles that then dissipate some of that adrenaline.

      If you get sweaty, that’s because your core body temperature is going up — it’s as if you’re exercising — the same thing’s going on: your heart’s beating faster, you’re tighter, your blood vessels are more constricted, your blood pressure goes up, your temperature goes up: You can cool yourself down, simply by holding something cold in the palm of your hand; your hands are <ooh!> thermal regulators for your body, just like uh your forehead and the back of your neck are.

      Sonal: Water bottle saves the day again. Water to the rescue.

      Matt: It does! So those are symptomatic relief. In terms of sources, so, many of us put a lot of pressure on ourselves to do it right. I’ve been doing this kind of work for three decades now, and I’m here to tell you, there is no right way to communicate. There are better and worse ways; if you can remove the pressure to do it right, you actually free up cognitive resources to do it better.

      So rather than seeing your communication as a performance where perfection is the goal, see it as a conversation where understanding and collaboration are the goal. And that takes a lot of pressure off of you.

      Now it’s very easy for me to say that, and it’s harder to do; but with work and practice, you can do that.

      Sonal: I have to tell you, one of the things that you’ve helped me with, as an anxiety- management technique, for big events and prep — one of the techniques you gave me is like having three keywords <mhm> as a way to kind of orient my identity before I go on stage. <right> And it is amazing how that helps me. And it’s funny, because my three words are: “energy, light, and shepherd”. <great!> And the reason is, because I’m a shepherd for the audience; energy, which goes to the point about feeling; and light, because I want people to feel enlightened — which I know sounds really mushy-gushy but, those are literally the three words that really ground that I’m collaborating with the audience: It’s not this oppositional, adversarial, dynamic.

      Matt: I think those are really empowering.

      Many of us are worried about a potential negative future outcome. The entrepreneurs that come to your firm are afraid they’re not going to get funding. My students are afraid that they’re not going to get a good grade. The people we coach to be better speakers are afraid they’re not going to get support for their ideas.

      That fear is a future fear. And because of that it makes it worse. So if you can short-circuit that, become present-oriented, focus on the moment, you by definition won’t be as nervous. So how do you do that? Well: Do something physical before you communicate, take a walk around the block. <Yes> You can listen to a song or a playlist; you see athletes do this all the time. The one I always joke about, but works really well: Start at 100 and count backwards by some difficult number. Try right now, start at 100 and count backwards by 17s… the only way you can do that is by getting really present oriented.

      Sonal: My therapist has given me a technique where, what’s the worst thing that could happen? <yeah> And we actually make it very concrete, it’s like oh, the worst thing that’s gonna happen is I run out of breath. <yes> And I’m not stopping breathing (which is what it feels like when you’re having a panic attack), and also, people are actually more friendly <right> then I think. And so all of that is extremely helpful.

      And knowing, it’s not weakness — but the better you know yourself, the better you can then plan and even reroute around or address it head on. I think a lot of times what happens is people deny it, they act like it’s something they have to run away from — because when you feel anxious, you just want to run away from the feeling, you don’t want it. But it’s far worse to be surprised by it on stage <that’s right> than to lean into the fact that you’re going to have it, so prepare for it.

      Matt: That’s exactly right. What I want people to take away is that — with practice, with commitment; giving yourself permission to take risks to try some of these things out — you can actually learn to be more comfortable and confident in high-stakes communication situations.

      Sonal: Matt, thank you for your time today. <chuckles> I hope this leaves everyone feeling empowered to be a moderator in whatever form. And, those that are more interested should go check out boldecho.com, your book, your podcast. And, I hope this is a helpful resource. Thank you for joining the “a16z Podcast.”

      Matt: Awesome.

      <fadeout>

      Sonal: Matt, there was so much insight per minute packed into what you just said…

      Matt: <chuckles> Well thank you… 

      • Matt Abrahams

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Textiles as Tech, Science, Math, Culture, or Civilization

      Virginia Postrel and Sonal Chokshi

      “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they’re indistinguishable from it.” That quote from computer scientist Mark Weiser is from a 1991 paper where he outlined the vision of ubiquitous computing; in it, he also referenced “seamlessness”… We just can’t get away from textile metaphors: we catch airline “shuttles”, we “weave” through traffic, we follow comment “threads” — the metaphors are as ubiquitous and abundant and threaded throughout our lives as the textiles (and computing) all around us.

      In fact, argues author and columnist Virginia Postrel, the story of textiles IS the story of technology and science (across all kinds of fields, from biology to chemistry); of commerce (as well as management, measurement, machines); but most of all, of civilization (vs. just culture) itself. That’s what her new book, The Fabric of Civilization: How Textiles Made the World is all about. But it’s really a story and history of innovation, and of human ingenuity… which is also the theme of the a16z Podcast — and of this special, inaugural book launch episode with the author in conversation with showrunner Sonal Chokshi.

      The discussion both dives deep and lightly dips into a wide range of topics: fabrics, from the genetics of cotton to the supply chain of silk (including pre-Industrial Revolution factories, early payment and incentive alignment, “maestre” and notions of expertise); knowledge, from the storage and transmission of it to sharing tacit and explicit code (including manuals, notation, measures); and math as the science of patterns, origins of mathematics (including early education and getting paid for it). The touch on the NASA space program, knitting and AI, and the environmental impact of dyes. Throughout, they discuss the what and the why — the warp and weft of this episode! — of HOW innovation happens, from incremental improvements to sudden leaps, also taking a closer look at the demographics and images involved. And finally, they cover the evolution and meaning of kente cloth (as well as other patterns) in Ghana and beyond… Because the story of textiles — and of technology — is not just a story of one culture or time or place: it is a universally human story, woven from countless threads and wires.

      Links & other articles mentioned in this episode:

      images: composite of knitting by © sarah-marie belcastro (courtesy Virginia Postrel) + magnetic core memory wires & beads, magnified 60x (photo from Virginia Postrel) — combined by Sonal Chokshi for the a16z Podcast

      Show Notes

      • Textiles as a metaphor for culture [2:20]
      • The cultivation of cotton and how it became a worldwide industry [9:24]
      • The ancient silk trade as a proto-industry [15:24] that required standard measurements [20:13], and a discussion of the critical role of the “maestra” [23:04]
      • The complex mathematics of textiles [25:24] and the science of knitting [31:00]
      • How information about textile creation was passed down through the generations [35:32], how we store ideas today [43:03], and a curious connection to NASA [44:26]
      • The ways that dyes led to modern organic chemistry [47:27]
      • Several case studies of innovation [49:57], and a discussion of traditional gender roles related to weaving [57:30]
      • The story of kente cloth [1:00:25] and how textiles are a part of global culture [1:07:35]

      Transcript

      Sonal: Hi, everyone. Welcome to the a16z Podcast. I’m Sonal. Today we have the very first episode — for a new book, coming out November 10, by Virginia Postrel: “The Fabric of Civilization: How Textiles Made the World” — which is all about the central role of textiles in the history of technology, science, commerce, civilization itself. But it’s really a history and evolution of innovation across all kinds of fields, so there’s something for everyone in this episode.

      Virginia has written several bestselling books including “The Substance of Style” — a long time favorite of mine — “The Power of Glamour,” which I excerpted when I was at Wired. She also wrote “The Future and Its Enemies” in the late 90s, and was the former editor in chief of Reason and has been a columnist for various magazines and newspapers, and currently she has a regular column in Bloomberg Opinion.

      But we’re here today to talk about her soon-to-be released book. In the long discussion that follows, probably [the] longest we’ve done here, we both delve deep as well as lightly dip into a wide range of topics. The first segment covers fabrics, from the genetics of cotton to the supply chain of silk, including early machines, early management techniques, what was considered expertise. We then cover the storage and transmission of knowledge, both explicit and tacit — including artifacts and manuals to notation and measurement — and especially mathematics — touching on NASA’s space program, knitting and AI, and the environmental impact of dyes

      And in the last section, we go “meta” on how innovation happens, from the zeitgeist of the times to the dynamics of codifying and sharing knowledge and the industrial enlightenment to the demographics and symbols involved. We actually go deep on the story of, evolution of, and meaning of kente cloth (and other patterns) in Ghana — to finally ending on the difference between cultures and civilizations, because the throughline that resonated most deeply with me throughout this book is that the story of textiles is one of human ingenuity — which is also what this podcast, the a16z Podcast, is all about. Please do also go ahead and rate us in your podcast app when you have a moment.

      And now, let me welcome Virginia. I’m so excited to have you on. Welcome.

      Virginia: Thank you.

      Textiles as culture

      Sonal: So, I’m not gonna ask you the obvious question, which is “why did you write this book?” First of all, I know why you wrote the book because we talked about it over many, many long dinners.

      Virginia: Yeah, right. I mean, this book wouldn’t exist without you.

      Sonal: One of my favorite things is you actually talk about all the metaphors we use, and you call them “heirloom metaphors” — but tell me about some of these metaphors. I mean, you basically say, “We catch airline shuttles, we weave through traffic, we follow comment threads,” like, all our listeners have done all of those things.

      Virginia: Right. Well, first of all, let’s start with “heirloom.” So, heirloom is a reference to a loom. And the reason that we have this word is that a loom was a piece of capital equipment. And so it would get passed down as a valuable thing. <Sonal: Heirloom, yes> Heirloom. But some of the more fun ones are ones that we have no idea. Like, “on tenterhooks.

      When people would make wool cloth, and I’ve seen this in a museum in England, after they would weave it, they would do what’s called “fulling” — which is a combination of wetting it, and pounding it, and using what’s called fuller’s earth on it — and it felted it. It made it thicker and more waterproof. And then, they would stretch the wool on these really vicious-looking little hooks — on a rectangular frame like nails sticking out — and when you see these tenterhooks, you realize, oh my goodness, that tension, that feeling of tension. I see why that metaphor came to be.

      Sonal: Perfect. So, your very first opening quote in the book is, “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they’re indistinguishable from it.” And that quote is from Mark Weiser, in “A Computer for the 21st Century,” which was, like, a paper he wrote in 1991. Tell me about why you started with that quote.

      Virginia: I love this quote for two reasons. The first is that it’s profoundly true. The most significant technologies are the ones that disappear. But the other is, you see what he did there?

      Sonal: Well, I see what you did there.

      Virginia: I don’t even think he realized what he was doing. Maybe he did. It proves my point, that this ancient, ubiquitous technology is really important. And, I want people to pay attention to it, and to pay attention to all the ingenuity that goes into the fabric that surrounds us.

      Sonal: A little-known fact about that Mark Weiser quote, and I remember this when I was at PARC, and I was writing up, like, some language for [a] living museum project, and one of the things that I found fascinating is we had these quotes about how he also said that not only are the most profound technologies the ones that disappear, but that sometimes, technology should be that when you leave a pencil behind in a meeting, you never go back and feel like, “Oh no, I have to go grab that pencil.” Like, you’re fine discarding that pencil. Like, it can be discarded. And, it’s just — that is what he thought of as ubiquitous computing.

      And of course, it’s the opposite with how technology has come out today in terms of, you know, we actually wanna keep our smartphones. They’re not like a pencil. But, the idea that something can become so ubiquitous as to be discarded, I love that quote also evokes that, which I don’t know if you’re aware that it did. But you yourself in the book talk about how “we suffer textile amnesia, because we enjoy textile abundance.” And I thought that was so profound.

      Virginia: Yeah, that’s perfect. I didn’t know that about Mark Weiser — who also, by the way, uses, in that article, the metaphor “seamlessness,” which is another textile metaphor. And this really has mostly happened in my lifetime (I’m 60), where textiles have just become so abundant and so cheap that we really don’t think about them. I mean, obviously I’m really contrasting it to a pre-industrial period, where, to make enough thread to weave the fabric in a pair of jeans would take something like thirteen 8-hour days. And that’s the fastest spinners in the world. And that’s just making the thread, that’s not preparing the fiber to make the thread, it’s not weaving, it’s not dyeing, it’s not any of the other processes.

      So, we live in this world where fabric is everywhere, and it’s really cheap — and, in fact, my friend Adam Minter has a great book called “Secondhand,” that’s in part about the secondhand-trade in clothes — among other things, also electronics. And so, we do take it for granted. We don’t think about where it comes from, and we don’t think about the labor, or the ingenuity over thousands of thousands of years.

      Sonal: This is exactly where there’s parallels to technology, because you point out — I never actually even thought about this — that the agricultural revolution was as much about fiber as it was food. But the Industrial Revolution did bring all these technologies, and as you point out, like, the hours that goes into making thread, the cost reductions and everything that that then enables — that’s exactly what’s playing out with technology right now, if you think about transistors being cheap and almost so ubiquitous, as to be wasted. So abundant, as to be forgotten, which is why I love the quote you open with so, so much.

      Virginia: So, driving down the same point, the other thing that I do in that intro is — Arthur C. Clarke famously said [that] any sufficiently advanced technology is indistinguishable from magic.

      Sonal: Yeah, it’s one of his three laws.

      Virginia: Right, exactly. And, what I say in the preface is, any sufficiently familiar technology is indistinguishable from nature. What I’m thinking about there is, we know intellectually that it’s not natural, that somebody made it, that there were machines involved, and chemistry, and all these things. But we never really think about it — so that’s what I was getting at. But there’s also — in the first chapter, which is about fiber — a very important point, I think, which is that there’s no such thing as natural fibers.

      All of these — what I would call biological fibers, like cotton and wool and linen and silk — are the product of many, many small, concerted, human interventions. They’re basically genetically modified organisms. In some cases, it’s extreme (like with silk). Others like cotton, naturally, [don’t] grow in most of the major cotton-producing parts of the world because they’re above the frost line. And so, if in that natural state, it would never make bolls. It would just freeze, and that would be the end of it. But, humans gradually modified it. And there’s sort of mysteries of why they would do that.

      The story of cotton

      Sonal: This is actually one of my favorite stories in the book, which is, you kind of go on this genetic sleuthing mission to figure out like the origination of how we were able to turn cotton into this more reproducible type of cotton, I guess, to grossly simplify it. I’d love you to quickly tell that story.

      Virginia: So, there are really two stories. One is a science story that’s about the origins of cotton, because there are around 50 cotton species around the world, and only one of them developed fiber, originally. And that was a species that geneticists called A, and it came in Africa. And then somehow — this is the great mystery — somehow it got to Mexico.

      Sonal: Did it float? Did it go on a boat? Like, what happened?

      Virginia: This is the mystery. So, it got to Mexico and it crossbred with a native species there that didn’t have fiber, and it produced what geneticists referred to as AD. The Mexican species was D. And this is what is known in biology as a “polyploidal” plant, which is common in the plant world. It means it has twice as many chromosomes — and so, therefore, there’s a lot more room to breed it, get more variety.

      Well, I interview this cotton geneticist named Jonathan Wendell, and he says [that] when he was first studying cotton genetics, there were two theories. One was that this crossing-the-ocean thing happened back when dinosaurs roamed the earth and the continents were still stuck together. And the other was — oh, well, human beings must have brought it on boats. It was what he calls the Kon-Tiki hypothesis, after the explorer Thor Heyerdahl early in the mid-20th century. But we now know from genetic research that it was neither of those — that somehow this cotton got from Africa to Mexico after the continents split apart, but long, long before there were any human beings. It could have gotten caught up in a hurricane. It could have gotten on a piece of pumice — we don’t know, but we can use these genetic clocks to bracket roughly how long it was. So, we owe cotton to this weird, random, unusual event.

      Sonal: And, by the way, so unusual that it hasn’t happened again. That was the point.

      Virginia: Right, it only happened once. It’s two things that only happened once. One was that a species of cotton developed fiber, and then two was that it crossed the ocean and developed this rich genetic code that could be manipulated in various ways.

      So, then people come along. And there’s old world cotton and new world cotton. In the old world, cotton was being raised in places like India that are in the tropics. But somehow we know that cotton spread with the spread of Islam — and it spread into places that were too far north for the cotton to grow. It spread in places that had frost. Why is that? And my theory is that because they wouldn’t have planted it there unless they already had cotton that would grow there — so, it must have come from places like India. And why would Indians raise cotton that could survive frost?

      Sonal: It’s a tropical climate, exactly.

      Virginia: It’s a tropical climate. They don’t need to. So, my theory is it might have been a commercial consideration. That essentially, if your cotton bloomed first, you could get to market earlier and get a jump on the competition. And so, over time, there would be sort of competitive pressures to get cotton to bloom earlier and earlier. And then once it bloomed early enough — like the cotton we have today — it would be good for planting farther north. Now, there is a competing theory, which is that it had something to do with controlling pests — but we don’t know. This is one of the great mysteries of cotton.

      So, nowadays, most of the cotton in the world (90%) is the species that originated on the Yucatan Peninsula, which is called gossypium hirsutum. And, it didn’t survive terribly well above the frostline late into the 19th century. And so I tell the story of this not-so-savory character who discovered a cotton in Mexico and he brought it to the American South, and that could grow above the frostline.

      Sonal: You know, there’s so many threads. And in fact, oh my god, I keep saying that. And I swear, it’s not a pun, Virginia. I keep saying thread and it’s actually a word I really legitimately use all the time.

      Virginia: Yeah, totally, you can’t — you can’t avoid it.

      Sonal: We’re just gonna notice it all the time now. So, a couple of quick notes on the story you just told about cotton. There are some details in your book about not only that Islam spread cotton, but you almost allude in that section, that maybe the adoption accelerated because of cotton. That’s one of the interesting ideas I noticed.

      Virginia: It was a desire to get out from under certain landowners and sort of strike out on their own. And there were ways you could do that by raising cotton that were connected with also converting to Islam.

      Sonal: So fascinating. And then the other thread, too — you talk about the British Empire in India, and people wanted Indian calico, but that the East India Company put limits on that. And then it kind of pushed other manufacturers like Italians to come up and step up to the plate. And then also — and this is so top of mind today, too — you talk about the ugly part of this history, that it’s the cotton gin, and history of slavery in the South, and even the observation that the South is not as untech-savvy as people portrayed — you know, compared to the “Yankee north.” So, I hope people pick it up and read it to find out more about this fascinating, terrifying history of all of that

      Silk supply chains

      What I want to focus on in this conversation — and this kind of follows how you organize your book from fiber, to thread, to cloth, to dye, to traders, to consumers — is you describe the evolution of what allowed us to mass-make cotton, and the steps into that. Let’s talk a bit about the supply chain, and vertical integration, and trade aspects. And that’s kind of a big jump — but I think it’s still a great jump to make.

      Virginia: Well, one of the things that is the case in all types of textiles is that you have a long supply chain. Well, you have a lot of stages, and there’s a long delay between when you, say, raise the cotton, and when somebody gets paid for the cloth.

      So, one thing that is the case throughout textile history is that you have to have a lot of innovation around working capital, and different ways of people getting paid. So, that’s one aspect. Textiles forced human beings to think, at a very early stage in civilization, about financial questions. About, how do you pay for things, where you’re providing inputs, rather than the final, finished product. Because you could raise sheep, and you shear sheep, and you spin wool, and you weave — but those things were not necessarily done in the same household <right>. In some ways, the most interesting developments around the supply chain take place in silk — for the reason that it is a luxury fiber, and also because it’s really complicated.

      Sonal: So, tell us about the silk and supply chain.

      Virginia: Okay. So, silk — I discovered while working on this book, a lot of people don’t know where it comes from, so let’s start there. Silk comes from silkworms (Bombyx mori), which are basically caterpillars that have been bred over many centuries, originally in China, to feed on mulberry leaves and produce cocoons. And the eggs need tending, the caterpillars need tending. They need to be fed, they need to be given sticks to build their cocoons on. And then, once they make the cocoons, they’re put into boiling water, which kills the insect inside, and keeps it from emerging as a moth and breaking — there’s a single filament that makes the cocoon.

      So, once you’ve done that, you need to get that filament off the cocoons. And so, you have skilled people who know how to take filaments off of multiple cocoons, and create thread. And there’s a lot of tacit knowledge involved in that. And that’s called “reeling,” because it goes on these big reels. And then often, you — when you want to make stronger thread, you have a stage that’s called “throwing,” where you twist two threads together to make a stronger one. And then you need to get that on bobbins, and that’s just the thread.

      So, one of the most amazing things that I discovered when I was working on the book, are these giant factories — 24/7, hundreds of people — that started in Italy, and had their heyday in the 16th and 17th century. So, this is before the Industrial Revolution — that was their heyday, but they started earlier — and they were organized around these very large, two-story throwing machines that were hydraulically powered. They would go down to the basement and there would be a source of water power. And these are all made of wood, they’re precision machines. It’s just amazing. There are museums throughout Northern Italy where you can see them.

      So, the machines are amazing enough. But it wasn’t just the machines. They developed sophisticated management techniques. First of all, they vertically integrated. They didn’t grow the cocoons. They went from cocoon, to thread for export, and then the thread would go to Lyon, which was the silk capital in Europe. I think most of us — and, I would include myself here — think about factories, and management, as starting with the Industrial Revolution. Maybe starting with Wedgwood. And this is actually before that. First of all, they are operating 24/7, and I asked a historian who covers them, like, “What did they do for light?” And he said they had torches. And I said, “That doesn’t sound very safe.” And he said, “No, it wasn’t.”

      And they developed measurement techniques, they developed standardized weights, they developed standardized tools for measuring things. One of the things they developed — which is used to this day — is the idea of measuring the fineness of thread by having a standard measure of length. And then how much the thread weighed, given that length. Which gives you some idea that ratio can tell you how fine the thread is.

      I mean, when you wander into the textile world, one of the things that’s really disorienting is they have all these weird ways of measuring things, like denier. What is a denier — you know, you may have noticed that when you buy stockings. And it’s from that idea of a certain amount of weight per — for a standard measurement.

      So, they developed a lot of techniques like that. And, this was proto-industry. So, then the question comes. Well, why isn’t that the Industrial Revolution? You know, why isn’t that the factory that changed? And the reason is that silk is a luxury. It’s a niche market. It can’t change the world. You don’t make sails out of silk. You don’t make sacks out of silk, you don’t clothe your armies with silk — all of these uses for other textiles don’t apply to silk. So a lot of techniques and machines were developed, and they did influence later developments, but they weren’t revolutionary in the way that transforming everyday textiles was.

      Sonal: Isn’t that so fascinating, because it’s such an inversion from, kind of, revolutionary tech today. Because, in that world, you did need mass — both for the capital outlays that you described, you know, that the financing and everything required — and the mass production of things. But in today’s world, the inverse is happening, where we have, like, sort of this late-niche, kind of more long-tail-driven economy in many ways. Because of the internet, of course.

      Virginia: Another way of thinking of it is, often new technologies start very expensive. I mean, who used original computers? It was big businesses. They were big, expensive pieces of capital equipment, and then they got smaller and smaller, and now we carry them around in our pockets. You sometimes have sort of luxury items as the first way into a technology because you can finance it that way, and justify [a] higher price. Now, I don’t wanna push it too far. That actually applies in terms of silk versus other things, more to the development of looms, and the drawloom, and Jacquard and his famous punch cards — the one thing that all technology people know about.

      The role of the “maestra”

      Sonal: So, you have this phrase, “management measure machines” — and those three words just popped out to me, because that to me is the art of technology and startups as well, or any business enterprise. But then you do point out a fourth M, which is the role of the “maestra.” Which, I didn’t know what that was. Tell us about the role of the maestra, and the — why that’s so significant in this supply chain and in this evolution from cloth to industry.

      Virginia: So, the maestra (that’s the singular), is a woman with many many years of experience, at reeling — that is, taking these incredibly fine — I mean, they’re strong, but they’re, like, less than a human hair — finds off of the cocoons, and twisting them together into a single thread. And the very best could do — make thread out of just two. And keep in mind, these are natural fibers, they’re biological. They are not totally consistent. So, one of the skills is matching — the matching up these fibers so that they are as consistent as possible, because that makes the best thread. So, it’s a really, really high-skilled job.

      Sonal: One might even call it pattern matching, but keep going.

      Virginia: Yeah, it is pattern matching. It is a form of pattern matching, along with a lot of manual skills as well. And you learn how to do it because you spend many years as an apprentice, being the person who turns the reel that takes up the thread, while the maestra is doing this and you’re watching her. So, you might spend 15 years doing that and acquiring all of that tacit knowledge. And these women were very well paid for the time, because they were making a luxury product. But, they thought about how to supervise people, they thought about incentives. So, for example, the maestre were not paid by the amount they produced, because they wanted to ensure quality — they were paid by the day, or by the time.

      Sonal: Oh, by the way, I underlined that because I thought that was such a great example of incentive alignment — like, modern management practice thinks about these very questions.

      Virginia: Right, right exactly. So, they were thinking about incentive alignment.

      The mathematics of textiles

      Sonal: So, you mentioned the ratio — and this sort of mathematical problem-solving that was already happening with the threading that the maestra did. The thinness, the thickness, etc. You also mentioned, you know, in the book as well, you open with this idea that technology means so much more than electronics or machines, which I love. And that is exactly what the book is about, but also, to me, the theme of this book — and in fact, of the articles that you did for Aeon magazine — the one that I set you up with Ross for — is that the story of textiles is one of technology, and science.

      And so, now let’s switch to talking about the math. You have this great quote, that “Spinning trains the hands, but weaving challenges the mind. Like music, it is profoundly mathematical. Weavers have to understand ratios, detect prime numbers, and calculate areas and lengths. Manipulating warps turns threads into rows, and rows into patterns, points into lines, and lines into planes. Woven cloth represents some of humanity’s earliest algorithms. It is embodied code.” I love this, Virginia. So, talk to me and tell me about the math of textiles and more of the story there.

      Virginia: First of all, I have to note that in researching the book, I learned how to weave, on a hand loom.

      Sonal: I did not know that. That’s so exciting.

      Virginia: It challenges me, because I’m not very good in three dimensions, and you have to think in three dimensions even though you’re producing two-dimensional stuff. It is kind of complicated. One point that I make in the book is that weaving is the original binary code — by which I don’t mean Jacquard and his punch cards, that came long long after — but, it’s all up, down, on, off, one, zero. That is woven into the cloth itself.

      And then also there’s this idea of math as the science of patterns. And I talk in the chapter about Andean weaving and the use of symmetry and — the people who are doing it aren’t necessarily thinking, I’m following algorithms. It’s sort of like the old thing about I’m speaking prose — but it is profoundly mathematical. And one of the really interesting speculative theories that I talk about is the idea that Greek arithmetic might have originated by the challenges of working on warp-weighted looms, because you have to know about prime numbers, especially on those types of looms, the techniques that they use.

      And another thing, which I didn’t get into [in] the book, Judy Frater, who ran the place that I went to India where I learned how to do printing and dyeing — she makes the point that in creating these complex printed fabrics in India, there’s a tremendous amount of ratios because you want to get the repeats exactly right. You need to be able to think mathematically to get things to look right.

      Sonal: That’s fantastic. One note in the later part of the book is, you talk about this 1976 study from Van Egmond. And it talks about “Abacus manuscripts” and books, and how he emphasizes their practicality. And the classical view of mathematics, inherited from the Greeks, was the study of abstract logic and ideal forms, whereas the Abacus books treat math as useful. Which I thought was so fascinating, because when I think of the fundamental challenge of learning and teaching mathematics — which is, like, my old world of work and research — that is the challenge, right there.

      Virginia: Yeah. So, first of all these Abacus manuscripts have a very misleading name because they’re the opposite of Abacus manuscripts. In fact, they’re teaching people how to do pen-and-paper calculation — not how to use an abacus. And there’s a history, which I explain about why they’re called that.

      But you had, in the early modern period, in Italy, the development of these schools, where people who were going to be merchants would go when they’re kids, to learn how to do arithmetic — sort of in between calculation and algebra. They do all these word problems, which, if you were solving them today, you would set up with variables and unknowns. And this was driven by essentially the need of textile merchants to do this new type of arithmetic, which was incorporating the Hindu/Arabic numbers and the zero, all of these things. They were moving away from using Roman numerals.

      So, these are business problems. A lot of them have to do with the currency conversions — with lengths, and if you get this much money for this length, and how much would you get for this length. All of this sort of thing. It’s the cloth trade driving a kind of mathematical education. And one thing that’s interesting is that these teachers were the first Europeans to make a living, just by math <Sonal: Oh right, that’s so interesting> they would teach these schools and hire out as consultants, particularly for various construction projects, where they would be doing more sort of geometrical calculations. So, that was driven largely by the textile trade.

      The science of knitting

      Sonal: I love that. I can’t even tell you, on so many levels. So, knitting is such an interesting thing to me, because a lot of people describe how knitting is technology. And so, I’d love for you to say more about the science and math of knitting.

      Virginia: Well, the interesting characteristics of knitting, mathematically, have to do with its three-dimensional qualities. So there’s an actual mathematical paper called “Any Topological Structure Can Be Knitted” — and there are pictures in the book of things like Klein bottles that have been knitted. From a practical, business point of view, this is very important, because we are at a technological moment where 3-D knitting — which has been around for several decades — is becoming more and more important. And it’s driven by the mathematical characteristics of knitting, the computing power, and the shift where, after millennia of being completely dominant, weaving is losing its market to knitting. Knitting is a relatively new way of making fiber. It started around 1200, whereas weaving goes back at least 9,000 years, probably more.

      Sonal: And why is knitting taking over weaving? That was so surprising to me.

      Virginia: So, it’s taking over weaving mostly because it’s more comfortable, because it stretches in multiple dimensions. Also, a lot of the athletic wear that drives technological innovation in the textile and apparel industries — athletic wear and outdoor wear — is knitted. A lot of it is just this drive for comfort, which has only been accelerated by the pandemic.

      Sonal: You make a funny note in the book about what you’re wearing while you’re writing it — and of course I chuckled because every time I met up with you, we‘d wear, like, you know, nice clothes.

      Virginia: Oh yeah, this is what I said. “Everything I’m wearing except my jeans — my underwear, shirts, sweaters, socks, even my sneakers — is made from knitted fabric.” But this incredible expansion of knitting, it feeds into the business side too, where — okay, we have this ability to do three-dimensional knitting. It has historically not been as economical as making big pieces of cloth and cutting and sewing, but it does allow us to make more to order so that your inventories are in thread rather than in finished garments or cloth. It allows more variety. There are advantages to it. And, you can knit an entire garment with no seams. Traditionally, most of the knitting that you have in your closet is pieced together, sewn together just like woven fabrics.

      Sonal: I don’t know if you realize that you said that the comfort-quality of knitting is that it’s essentially seamless — and that, of course, goes back to the Mark Weiser quote — because earlier you said that the other word in his big paper was seamless.

      Virginia: It can be seamless — and that is, in fact, what SHIMA SEIKI, who is the company that first innovated this three-dimensional knitting, although they’re not the only company doing it now — they sold it as “the seamless garment.”

      Sonal: Got it. And by the way, because our listeners are very curious — and this is what I love about everyone — the paper, if they’re interested, that you referenced, is “Every Topological Surface Can Be Knit: A Proof.” And the journal, the full name is “The Journal of Mathematics and the Arts,” and it’s from June 2009, and the authors are Sarah-Marie Belcastro and Carolyn Yackel. And oh, it was also in “Math Horizons” in November 2006. So, those are the sources. Oh, by the way, on the 3-D notion of knitting, have you heard of this project called SkyKnit — which is basically a neural net that generates new patterns for knitting; have you read about this?

      Virginia: I haven’t, no.

      Sonal: Oh, you would love this — I’m gonna send you an article about it. It’s on Janelle Shane’s work, and she, you know, does a lot of creative AI-type experiments. But, she basically took the knitting forum Ravelry, and she trained a neural network on a series of 500 knitting instructions, and then she generated new instructions. And that the Ravelry community has actually attempted to knit. It’s just one of my favorite stories of online things.

      Virginia: I’m not surprised it exists.

      Sonal: One of my other favorite stories is, and this is an Instagrammer I follow who’s Shannon Downey, because I’m super into all the fabric textile artists — and she basically had heard about this unfinished quilt, and that, you know, they found a box full of fabric and discovered it was a massive quilting project that was just begun, and it was mapped out. And so, so many people on the internet joined up to quilt this quilt — and it’s called Rita’s quilt — and they just completed it.

      Passing down techniques

      In that case, it’s more technology for collaboration than for creation, but it’s a segue to my next question, which is, let’s talk about the artifacts — and really, this theme of record-keeping and the transmission of textile knowledge. That’s, like, another really interesting theme in your book, and so I’d love to probe on that a bit.

      Virginia: So, traditionally, textile knowledge is transferred through apprenticeship — either formal (like, there were these apprenticeships in Europe, they were highly regulated and such) or informal (within families, or villages or whatever). But you have a shift in the early modern period, in Europe — there’s more documentation in China earlier, but you have a shift in the early modern period, where there’s this idea in the air that it’s a good thing to share knowledge, as opposed to keeping it secret.

      Sonal: Yeah, it’s in the zeitgeist.

      Virginia: It’s in the zeitgeist, and it’s very important in the history of technology and science, and the touchstone example is Diderot’s Encyclopedia and — which, in fact, includes a lot of documentation of how various looms work, and [the] source of some of the illustrations in my book. But in my research, I found a couple of less well-known examples. One is the first dye manual, which is called Applicto. Iit seems to come from some word for envelope. And essentially you had an Italian guy, who went around over a long period of time, prizing various recipes out of dyers, and he published the first dye manual — sort of like a recipe book. It tells you [to] use this much of this or this much of that — so that’s one.

      And then the other one — which is very interesting, because it reminds us how important notation is — is the first weaving manual. Nowadays, if you go online, Ravelry has these weaving “drafts,” which is a form of notation that tells weavers how to set up a loom, which threads to raise, in what order to create a pattern. Well, somebody had to invent that notation. And, probably it was invented for personal use, and it was secret. But there was a guy who was very frustrated that they would have to import textiles to get the good patterns, and he was, like, “No, we can do just as well. The problem is people are too secretive about the knowledge, and they aren’t teaching other people.” So, he put together a manual that was the first weaving manual — and there is a site called handweaving.net, where they’ve turned a lot of these instructions into instructions for today.

      But this is a theme throughout the book, that there is a tension — because making textiles requires a lot of tacit knowledge, a lot of the kind of knowledge that’s very difficult to write down and articulate — we talk about the maestre reeling cocoons, I mean, it would be very hard to convey that to another person. But there are other types of knowledge that can be codified, and so there is this move toward codification. That really helps jumpstart a lot of technological progress, because you’re able to have people from outside fields to understand what’s going on in other fields, and maybe apply that in other ways. Or you just get people better able to understand the state of the art and technology, and perhaps innovate on top of that. And then the notation that develops for organic chemistry is really important also. I don’t use it in the book, but I do talk about the development of organic chemistry out of the dye industry.

      Sonal: It was funny, because when all my friends in college were taking O-Chem — you know, like, shorthand for organic chemistry, I remember just kind of enjoying the notational aspects of it. So, basically the notation, the manual encoding helped make the notation more standardized and common as well, and then there was a zeitgeist move of making the knowledge and artistry of weaving more public. You quote Ziegler, who did that original weaving handbook, that “I hold that it would be possible to produce many more artists in all branches of technology, if only there were no shortage of publishers.” Which I loved, because what it told me is that that manual was basically the printing-press moment of the textiles industry.

      Virginia: Exactly, and this resonates so much with today in terms of all the riches we have available online — which includes not only printed stuff but videos, which are incredibly important in people learning how to do things — but it also goes to the tensions that we have around copyright. You know, the history of textiles is full of industrial espionage. I have a few examples in the book, but people were constantly trying to keep stuff secret, and other people were trying to get it, because it’s very valuable.

      Sonal: I’m glad that you share that in the book. And I don’t know if you know this, but I actually interviewed Joel Mokyr on this podcast a number of years ago. Yeah, the podcast is called “Knowledge builds technology and technology builds knowledge,” and it was about the Republic of Letters, and that — its role in what he calls “the Industrial Enlightenment.”

      Virginia: Oh, and I use that term a lot. Joel’s work is really great, and I’m very influenced in my understanding of this zeitgeist moment by his work. Because you have this moment where you have craftsmen, intersecting with codification, intersecting with scientists — and, each feeds the other in ways that were not previously happening. And that really allows a certain kind of scientific and technological takeoff — particularly technological takeoff — because it gives people where to look for new advances — and also just shares things across borders, shares things across class. You get to tap a lot more knowledge.

      Modern-day idea storage

      Sonal: You know, one of the things that fascinated me on the history of the knowledge transmission of patterns and record keeping is the idea of storing things in song, and in the cloth itself, and you talk about that in the book, and I thought that was so beautiful, yet so ephemeral. And so, one thing I wonder when you talked about how it’s really important that we now have videos for so many — do you worry about the artifacts of the future, and how future historians might come back into our time and look at some of this?

      I don’t know if you have any thoughts on this, but I was intrigued by this idea, simply because you open the book talking about how over half of the tablets that were uncovered on Crete, when they were doing excavation of the ancient minotaur labyrinth area, they were textile tablets. And will future generations have those concrete artifacts? I’m very seduced by that idea of ephemerality and permanence.

      Virginia: Well, I am a great supporter of the internet archive. I think it’s an incredibly important organization.

      Sonal: We actually had Brewster Kahle on the podcast.

      Virginia: It’s incredibly important. So, do we lose this kind of traditional knowledge? One thing that’s interesting is, since about the 1970s, there’s been a tremendous appreciation and recording of people’s traditional knowledge about traditional crafts. And what interests me the most about that is not just that it happened — and there’s some really interesting stories about how it happened in different places, particularly Peru — but also that when you have a living textile tradition, it’s very different from what people in developed countries sort of think it ought to be. It’s not static. It’s not producing the same thing that the ancestors produced. It’s subject to all the kinds of changes that affect all of us.

      And you have people adopting and adapting their traditional techniques for the world they actually live in, which is the 21st century world. And I tell some stories from Guatemala, and I’ve also written about some stories from Chiapas in Mexico that [were] not in the book. It’s in an article, it’s online — all these things are on my website also.

      Sonal: Great, I’ll link to some in the show notes as well. One last question on this thread. I do think it was interesting that you talked about how programmers first wrote code using punch cards, and this idea of how NASA produced “rope memory.”

      Virginia: So there are two intersections between early computer memories and weaving. The first, which was the dominant form of computer memory, before the development of silicon chips, was core memory. And essentially, what you had was weaving. You had threads, copper wires, going horizontally, intersecting with ones going vertically, just like cloth. And then at each intersection, you would have a ferrite donut — they were tiny, tiny, tiny — and, by sending a certain electrical signal down the right thread, so to speak, you could flip the core from positive to negative, and it would be a one or a zero. And that’s how essentially RAM was done.

      Sonal: RAM, as in Random Access Memory, right.

      Virginia: Random Access Memory. That was before silicon — that was, like, from the 50s to the 70s. That was how computer memory for most purposes was done. In the Apollo program, they needed to do essentially, ROM — Read Only Memory — in a very stable and very compact and lightweight form. Relatively lightweight. And essentially what they did was called “rope memory.” Instead of having the intersection of the warp and weft, so to speak, with a core that could flip, they would just — they would write the program on punch cards, they would debug it, they would get it all working, and then they would code it into wires that either went over or under, depending on if it was a one or a zero. And so, it was literally software you could hold in your hand. It was this physical embodiment — I mean, I guess the punch cards are a physical embodiment too, but this was much more compact. And that was used in the Apollo program.

      Sonal: What is rope memory but the code that runs the space program? It’s a perfect thread to end this section of how to store knowledge, transmit knowledge. I also love — throughout your book, there’s actually a few semiconductor analogies smattered about. I finished Andy Grove’s “High Output Management” book recently for the first time. People have been talking about it for years, and I finally read it this year, and I was so struck, though, by how much semiconductor manufacturing actually applies to creative work, too. And it’s just so fascinating, when you see all the analogies from semiconductors to textiles as well, and in some of these creative crafts as well. So, I just think that’s fantastic.

      Dyes and modern chemistry

      We don’t have time to go into the dye chapter. You already pointed out how the dye industry was one of the earliest applications of some of the organic chemistry notation. But the one interesting fact that I wanted to quickly flick on in this chapter, which was really counterintuitive and quite surprising to me — especially because Marc, Andy McAfee, and I did a podcast on his book “More From Less” is that — people have this assumption that the past was more environmentally conscious than now. And in fact that’s not true.

      Virginia: Yeah. One thing that people don’t realize when — particularly when they talk about “natural dyes,” is — they’re very messy, very smelly. Indigo particularly, which is a wonderful, wonderful dye. But it stinks. And it’s not even the stinkiest dye that I talk about in the book. So there’s that. And then the other thing is dyeing uses a lot of water. And I was really struck by this when I took dyeing classes in India, in Adipur, which is in Kutch, which is a desert area, so —

      Sonal: It’s in Gujarat, yeah.

      Virginia: Yeah, it is <It’s my home region>, so it’s not like water is plentiful. But I came from California, Southern California, and we were in a drought, and so I was hyper conscious of the use of water — and they’re just using tons of water, throwing the water in the yard, throwing the water in the yard. And this was not particularly polluting, but it really struck me — the combination of the history of dyes as this kind of thing that you wanted outside town — people didn’t want to be next to dyers — with this use of water, even today.

      And I think when we think that in the olden times, everything was environmentally benign, that’s partly a matter of scale. When you have large-scale production, you’re gonna use more of everything. But it’s really the very modern plants — and I talk about a dye house I visited in LA — that are using really precise measurements. And being — looking for innovative ways to save on water, save on energy, save on time, save on labor, just the — all the economic factors, as well as the environmental factors — that’s where you really get the innovation that becomes environmentally more benign. It’s not by going back to the past.

      Sonal: It’s sort of this yearning for a past that never was.

      Virginia: Yes, exactly.

      Textile innovation over time

      Sonal: And the technology is the way forward, and if we do want to do a lot of these great things. So, now some kind of meta theme questions around innovation. So, you actually have a chapter at the end called “innovation,” which made me chuckle because, actually, your entire book is about innovation.

      Virginia: The whole book is about innovation. Actually, my editor — I had a great editor, but her idea, we were talking about this last chapter. She said, “Why don’t you call it ‘innovators’” — which is parallel to traders and consumers — but there are innovators in every chapter.

      Sonal: It’s not orthogonal, it’s both the warp, the weft. So, let’s kind of come full circle. You say that “to weave is to devise, to invent, to contrive function and beauty from the simplest of elements.” And I thought that was profound, because I think that is exactly what innovation is. It’s transforming something into something else, or taking building blocks like Legos, or words, or code, and turning them into something else. Or Chris Dixon, in his post, described how code “is the encoding of human thought” — and in your book, you actually have a line in the the chapter on fiber, where you talk about Hardy’s work, and how when you twist things into forming ropes and knots, that it “demonstrates an infinite use of infinite means and requires a cognitive complexity, similar to that required by human language.”

      So, I want to ask you about more of the how versus the what — which is, you say that this is a story of innovation, things that are both famous and forgotten. Incremental improvements and sudden leaps, repeated inventions, and once-forever discoveries. So, I’m gonna have you do — very lightning-round style — give me an example, in the meta story of innovation, of an example of something famous, even still famous today.

      Virginia: The invention of nylon by Wallace Carothers. Nylon is the first polymer fiber, and in fact I — Carothers is the person who figured out what polymers, which also come in protein forms — he figured out that these really were these giant molecules and he demonstrated it. But nylon is the first synthetic fiber, and the first polymer. And I tell the story. The inventor is less famous, but nylon is famous.

      Sonal: And give me an example of a forgotten — I mean I guess your whole book has a lot of these, but a forgotten one.

      Virginia: So, the person that I would really love to know who invented it is, somewhere in China, there was probably a woman [who] was involved in silk production who figured out what we now know as the belt drop off. And the particular application was what is called, in spinning, a spindle wheel, as opposed to spinning wheel. She figured out how to take a big wheel, and put a thread or a belt onto a little wheel — and you could turn the little wheel many times, by turning the big wheel. Basically what this person figured out was you could turn that not-load-bearing wheel on its side, and attach it to another wheel, and then you could have it turn faster. And this is incredibly important in the history of textiles, and it’s incredibly important in the history of machines — and we have no idea who did it. Although we have a reasonable idea of what kind of person it was and where it was.

      Sonal: Okay, so that’s “famous” and “forgotten.” So, now let’s do “incremental” and “big leaps.” So, give me an example of an incremental improvement that was important in advancing the fabric of civilization.

      Virginia: All the incremental improvements in the quality of cotton for example, in not only breeding cotton that could grow above the frost line, but making it more disease resistant, making it — longer fibers, more per boll. All of those kinds of incremental improvements.

      Sonal: Now, give me an example of an opposite of incremental improvement — a “sudden leap.”

      Virginia: The invention of aniline dyes, synthetic dyes, which set off essentially organic chemistry.

      Sonal: Say a teeny bit more on that one.

      Virginia: Yeah. So, there was a 19-year-old student called William Perkin, who was fooling around — he was trying to actually synthesize something that would be like quinine, a malaria drug. And he didn’t get what he wanted, but he noticed that this precipitate in his beaker was kind of purple-y, and he tried dyeing silk with it, and it worked really great. And then he said, hey this is a business — and like many many entrepreneurs before have said, if he’d known what was going to be involved in taking his bright idea and turning it into something that could actually be a company, he might never have done it — but he did. Because they had to invent all kinds of machinery, and ways of producing large chemical sources, stuff like that. <I love that> And there’s actually a whole book about that called “Mauve,” but I tell the essentials of this story.

      Sonal: That’s a story that I remember learning from Xerox PARC, which is, when you have material-science type innovations — like, the piece, the component is one tiny, tiny piece of the puzzle — it’s actually the broader ecosystem that you build around it that’s actually the true innovation. And I was on the frontlines of watching a lot of the architecture and orchestration and attempts at that.

      Virginia: And this is also why — I mean, a lot of textile innovation falls under material science today. And this is why people publish about some cool new textile idea — and it never sees the light of day. Because it’s not viable, either as a business, or in some cases just as an actual textile. And I tell some stories about the 20s and 30s, things that people were fooling around with. Because rayon was around, and they had this idea that you could take natural materials and transform them — because rayon is made basically from wood — and turn them into fibers. And so, they were trying, like, everything. Eggs, and milk. And I talk about the milk fibers, which were very big in Italy, very backed by Mussolini’s government and stuff, and why they didn’t maintain themselves after the war.

      Sonal: Well, that goes to the next lightning-round one, which is repeated inventions. Because today, there is a resurgence in kind of wood-based and, like, bamboo-based fibers as alternatives to cotton, for instance, and other fibers. So, what would you put on your list of a repeated invention — something that was invented over and over again many times throughout the history of textiles?

      Virginia: Well, the simplest one is the drop spindle, which is essentially a weight with a hole in it, with a stick. And the weight maintains the angular momentum and lets you draw out fiber and spin it. And it was invented in slightly different forms, all around the world.

      Sonal: And now, what is an example of a — not a once in a lifetime, but once in forever kind of discovery. And it doesn’t have to be invented by human ingenuity, it could be discovered by human ingenuity.

      Virginia: I mean, certainly we talked about the idea that cotton only happened once, but that’s not really human ingenuity.

      Sonal: That’s why I was giving you an out, actually, because they did discover…

      Virginia: Well, I actually used my “only once,” which was the spindle wheel. <Right> Which you would think with all this spinning all around the world, you would think that people all around the world would have figured out ways to do it faster, but it actually only happened once. It happened in China, and then it spread from there.

      Gender roles in portraits

      Sonal: It’s interesting that you speculate that it was likely a woman. The thing that actually struck me personally, is this idea of the role of women and taking back the power of, sort of, women as some of the original coders or original scientists. A lot of critics today tend to emphasize that there’s this implied domesticity and subordination of the era’s images of spinning women, as in doing weaving. And you talk about how that may not necessarily be the case.

      Virginia: Right. In the chapter on spinning, I start with a paired set of portraits that are in the Reich’s Museum, in Amsterdam. And it’s a husband and wife. The woman is spinning, and the man is a businessman, he’s got his account books, he’s got some money — symbols of their trade. And these are actual, real people. We have a pretty good idea who they were, they’re not types — but these could also be the iconographic symbols of industry and commerce. Before smokestacks became the images of industry in the, say, 19th century, <right> the symbol that people used to symbolize industry was a woman spinning. Because this was the epitome of sort of productive labor that’s making something, as opposed to commerce, which is trading something.

      And so, you would have these very common images — and that’s because it was an incredibly important activity. Cloth consumes enormous amounts of thread, and there was never enough thread. That is why, in fact, it made such a big difference when you have the Industrial Revolution. Why does it start with thread? Because thread is incredibly in short supply always, and it’s this input into all these different kinds of uses of cloth. Everything from sacks to sails for the British navy, to all of these kinds of things. Really important. And so, women’s work — it was domestic, but then a lot of men’s work was domestic, too, and it’s not a symbol of subordination in these portraits. It is a symbol of equality. And in fact, if you look at the construction of the portrait, in sort of a deep art-analytical sense, like, their hands are in exactly the same positions. It’s commerce and industry as equally important.

      Sonal: And by the hands, you mean that there was a certain curl. You describe this in the book…

      Virginia: Yeah, exactly, yeah.

      Sonal: …that shows that it wasn’t just someone posing, it was the precision of an actual weaver who knew the art.

      Virginia: She is actually positioned like she knows the art. And then his hands are more artificially posed, with the coin and with the book and stuff — but they mirror the positions of her hands.

      The unique case of kente cloth

      Sonal: So then, the last tension is that you open the book talking about how this is so ubiquitous across cultures. And in fact, when you describe the words that we use — you talk about French words, like métier. You talk about non-European words, like in the Mayan language — the terms for weaving designs and hieroglyphics both use the same root. In Sanskrit, the word for sutra, which now refers to, like, a religious scripture, it originally denotes a string or a thread. You talk about the word tantra, which is tied to tantrum or warp or loom. The Chinese word zǔzhī, meaning organization or arrange, which is also the word for weave. And chéngjiù, which means achievement or result, which originally meant twisting fibers together. So basically, you really outline how this is so ubiquitous throughout multiple cultures.

      So, now I’m gonna just have you tell me one last story, which is the story of kente cloth. I think what’s really fascinating to me about the kente cloth chapter is my mom [was] born and raised in Uganda — which is not originally where kente cloth is from, right — my whole entire family’s from Uganda. <Oh right> So, I grew up…

      Virginia: Did they come to the U.S. after they got kicked out…

      Sonal: So, it was the Idi Amin. Exactly.

      Virginia: I remember when — I remember when that happened, yeah.

      Sonal: It was — it was before I was born. My grandparents, her parents got relocated to the U.K. and my aunt went to Sweden. But, I grew up surrounded by African fabrics. And then we have this political moment, where people are using kente cloth as a symbol. The thing that I wanna focus on — and especially, by the way, because it also to me tied together your other books, and that includes both “Glamour” and particularly “The Substance of Style,” where you talk about…

      Virginia: It was very resonant on that one, yeah.

      Sonal: It’s very resonant on that one, because you talk about how fashion is totemic.

      Virginia: So, West African weavers have for a long, long time, made cloths that are called “strip cloths” — which is that they weave 4-6” wide strips, which are then sewn together into a complete cloth that’s then wrapped around the body in various ways. And this is ancient. And in fact, I talk in the book about how strip cloths were even used as currency.

      Sonal: Cloths as coin, that was so cool.

      Virginia: Kente cloth is actually fairly recent — dates back just to the 18th century, probably, 19th century. We don’t know exactly when it originated. And in Ghana, there’s a great contestation between two ethnic groups — the Asante and the Ewe — about who invented it, because it’s the national cloth and source of great pride. And I talk about the research that’s been done that suggests that its distinctive weave pattern was probably developed by the Ewe, but quickly adopted by the Asante, who then put on their preference for bright colors, which is very much what we associate with kente cloth.

      And what makes kente cloth distinct is not the patterns, although they are distinct and interesting — it’s the way it’s woven. It is woven in an alternating sort of squares of warp-faced and weft-faced weaving, so that it’s something — you could not produce a total kente cloth on an industrial loom. It has to be woven in these strips, and those strips have to be put together, and there’s a special loom that was invented to make this kind of pattern.

      Kente cloth is very interesting technologically, or artistically, because it requires a great deal of forethought and planning. So, that’s interesting. Now, what makes it particularly interesting — and, of course, resonant in the U.S. — is how it went from being a kind of cloth for particularly Asante aristocrats, to then when Ghana became independent and you had the sort of pan-African idea, it became a symbol of sort of Africa in general, the African diaspora. Muhammad Ali went to Ghana and he wore kente cloth, it was a big deal. The president of Ghana was photographed in “Life” magazine shaking hands with Dwight Eisenhower on a state visit to the U.S., and he was wearing kente cloth. And in moving from Africa to the United States, its meaning and even its physical characteristics changed. First of all, the Asante elite were not related to the ancestors of African Americans. Their ancestors were the ones who sold into slavery. But textiles mean what people want them to mean, how people use them, and their meanings evolve.

      And so, what you had was — first of all, you had this transformation in the 1960s of kente cloth from being a symbol of a tribal elite and a national elite, to being a symbol of African pride and the African diaspora. And then you started to have people who were famous — like W.E.B. Du Bois, they would put a little kente cloth in their stoles that they wore with their academic robes. And then, at Westchester University of Pennsylvania, students adopted, as black students, to symbolize their pride in their academic achievements and their racial identity. And they started wearing not a whole piece of kente cloth, but essentially something that amounted to one of the strips that it was made from. And this spread to becoming a widespread custom, particularly at graduations. And this is what a living textile tradition does. Because people want many different things out of textiles, and one of them is an expression of their identity, and also who they want to be, who they aspire to be.

      And then you have kente cloth also go from being always a woven pattern, into being frequently a printed pattern for all kinds of things — where it takes on this, kind of, new meaning and new purpose. So, kente cloth has had many different incarnations, and it has different incarnations in Ghana. I mean, the top weavers of it make new patterns all the time.

      Sonal: There’s a Ghanaian fabric maker that created a fabric featuring the Ghanaian president — because he made lots of frequent televised speech appearances during the pandemic — printed spectacles, you know, floating against, like, the swirling red, white, and green background, which I thought was fabulous. And it even had symbols like padlocks for signifying lockdowns, and plane propellers for the borders that are closed. It just goes to show how culture evolves so quickly, in so many ways, and people find meaning. I’ve also been fascinated growing up, you know, going to India, that patterns — the history of patterns, like, ikat. Like, it’s part…

      Virginia: Oh, I love ikat.

      Sonal: I do too. And it’s widely used in Turkey. But then some of the techniques are, you know, rooted in shibori in Japan. But shibori itself was something…

      Virginia: They have their own kind in Guatemala.

      Textiles and world culture

      Sonal: Oh, I love that. But the other thing that I think is really beautiful about this, is it’s not only a story of things being reinvented or invented multiple times all across the world in different ways — but it’s also a story of borrowing, and reborrowing, and building on — because you know, shibori started in India and it went back to Japan, and it went back to China. I mean, it’s just like all over the place. I just got that all messed up, but the technique just kind of comes back. And last time I went to India, it was totally in vogue to buy, like, salwars that had, like, modern shibori in cool new colors like neon. Which I think is so cool. Like, this is the story of culture and technology. But you also talk about how this is actually not about culture, but about civilization, and how there’s actually a difference between the two.

      Virginia: You can have multiple cultures within a civilization, and you can have a continuous civilization where the cultures change. And I’m not using it in a, like, “civilization good, barbarism bad” kind of way. I’m using it to describe, essentially, a survival technology that humans have as a way of transmitting ways of protecting themselves against [a] hostile world and hostile people.

      And then the second idea of it is that it’s cumulative. It builds, and that cumulative nature can be broken, or it can continue. The example I use is, think about Western Europe in 1980. And think about Western Europe, or as it was known at the time, Christendom, in 1480. That is a continuous civilization, but it culturally couldn’t have changed more. The politics were different. The people’s understanding of the natural world was different. How they dressed was different. How they spoke was different. Every aspect that constitutes culture had significantly changed, but this cumulative civilization had continued. And what I’ve argued in a new article out in “Reason,” where I develop the idea about civilization in the current context, is that some time in the recent past, we — for the first time ever — developed a world civilization. We had a coming together of east and west into a single civilization that hadn’t been the case ever in human history before.

      So, when I talk about the fabric of civilization, I’m talking about the continuous building up of knowledge, and techniques, and technology that helps knit together — to use a textile metaphor — different civilizations.

      Sonal: So, this is a great note to end on, because while you talk about how this book is about the fabric of civilization — which is cumulative, it has layers, it’s about survival, you know, cloth protects us — what you end the book on in your afterward is this beautiful quote, which is that “This heritage does not belong to a single nation, race, or culture. Or to a single time or place. The story of textiles is not a male story or a female story. Not a European, African, Asian, or American story. It is all of these — cumulative and shared, a human story, a tapestry woven from countless brilliant threads.” And that is the essence of this book, “The Fabric of Civilization: How Textiles Made the World.” By Virginia Postrel. Thank you for joining this episode of the a16z Podcast.

      Virginia: Thank you.

      • Virginia Postrel

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Data Alone Is Not Enough: The Evolution of Data Architectures

      Ali Ghodsi and Martin Casado

      Data, data, data — it’s long been a buzzword in the industry, whether big data, streaming data, data analytics, data science, even AI & machine learning — but data alone is not enough: it takes an entire system of tools and technology to extract value from data.

      A multibillion dollar industry has emerged around data tools and technologies. And with so much excitement and innovation in the space: how exactly do all these tools fit together?

      This podcast — a hallway style conversation between Ali Ghodsi, CEO and founder of Databricks, and a16z general partner Martin Casado — explores the evolution of data architectures, including some quick history, where they’re going, and a surprising use case for streaming data, as well as Ali’s take on how he’d architect the picks and shovels that handle data end-to-end today.

      Show Notes

      • The history of data storage architecture, from data warehouses to data lakes [1:20]
      • Whether analytics and AI/ML should be thought of as two separate markets [3:35], and a discussion of using SQL to accommodate AI/ML [5:28]
      • Use cases for real-time and streaming data, and a discussion of latency [10:33]
      • Best practices for designing a modern data stack [13:17] and predictions for the future of data architecture [20:05]

      Transcript

      A brief history of data architectures

      Ali: It kind of started in the ’80s. Business leaders were flying blind, not knowing how the business was doing, waiting for finance to close the books. This data warehousing paradigm came about where they said, “Look, we have all this data in these operational data systems. Why don’t we just get all that data, take it out of all these systems, transform it into a central place, let’s call it a data warehouse, and then we can get business intelligence on that data?”

      And it was just a major transformation because now you could have dashboards. You could know how your product was selling by region, by SKU, by geography. That has created at least $20 billion market that has been around for quite a few decades now.

      But about 10 years ago, this technology started seeing some challenges. One, more and more data types, like video and audio, started coming about, and there’s no way you can store any of that in data warehouses.

      Second, they were on-prem big boxes that you had to buy. And they coupled storage and compute, so it became really expensive to scale them up and down.

      And third, people wanted to do more and more machine learning and AI on these data sets. They saw that we can ask future-looking questions. “Which of my customers are going to churn? Which of my products are going to sell? Which campaigns should I be offering to who?”

      The data lake came about 10 years ago. And the idea was, “Here’s really cheap storage, dump all your data here, and you can get all those insights. And it turns out, just dumping all your data in a central location, it’s hard to make sense out of that data that’s sitting there. As a result, what people are doing now is they’re taking subsets of that data, moving them into classic data warehouses in the cloud.

      So, we’ve ended up with an architectural mess that’s inferior to what we had in the ’80s, where we have data in two places, in the data lake and in the data warehouse, where the staleness and the recency is not great.

      In the last two to three years, there’s some really interesting technological breakthroughs that are enabling a new kind of design pattern. We refer to it as the lakehouse. And the idea is: what if you could actually do BI directly on your data lake? And what if you could do your reporting directly on your data lake, and you could do your data science and your machine learning straight up on the data lake?

      Analytics & AI/ML: one market or two?

      Martin: I would love to tease apart a few things that have led us here. There’s very clearly a large existing data warehouse market around BI and analytics, typified by people using SQL on structured data.

      It seems like the ML and AI use case is a little bit different than the analytics use case. The analytics use case is normally human beings looking at dashboards and making decisions, whereas in the ML/AI use case, you’re creating these models and those models are actually put into production and are part of the product. They’re doing pricing, they’re doing fraud detection, they’re doing underwriting, etc.

      The analytics market is an existing buying behavior and an existing customer. ML/AI is an emerging market. And so the core question is: are we actually seeing the emergence of multiple markets or is this one market?

      Ali: There are big similarities, and there are big differences. Let’s start with the similarities. Roughly the same data is needed for both. There’s no doubt that, when it comes to AI and machine learning, a lot of the secret sauce to getting really great results or predictions comes from augmenting your data with additional metadata that you have.

      In some sense, we have the same data, and you’re asking analytical questions. The only difference is one is backward-looking, one is future-looking. But other than that, you want to do the same kinds of things with the data. You want to prepare it. You want to have it so that you can make sense of it. If you have structural problems with your data, that also causes problems for machine learning.

      The differences today are that it’s a line of business that’s typically doing AI and data science or hardcore R&D. Whereas data warehousing and BI oftentimes sit in IT. Users of the data warehouse and the BI tools are data analysts and business analysts. In the case of machine learning, we have data scientists and machine learning engineers. So, the personas are different and sit in a different place in the organization. Those people have different backgrounds, and they have different requirements for the products they’re using today.

      But can’t we just use SQL?

      Martin: If you talk to some folks that come from the traditional analyst side, they’ll say, “AI and ML is cool, but if you really look at what they’re doing, they’re just doing simple regressions. Why don’t we just use the traditional model of data warehouses with SQL, and then we’ll just extend SQL to do basic regressions, and we’ll cover 99% of the use cases?”

      Ali: Yeah, that’s interesting that you ask because we actually tried that at UC Berkeley. There was a research project that looked at: Is there a way we can take an existing relational model and augment it with machine learning?

      And after five years, they realized that it’s actually really hard to bolt machine learning and data science on top of these systems. The reason is a little bit technical — it just has to do with the fact that these are iterative, recursive algorithms that continue improving the statistical measure until it reaches a certain threshold and then they stop. That’s hard to implement on top of data warehousing.

      If you look at the papers that were published out of that project, the conclusion was we have to really hack it hard, and it’s not going to be pretty. If you’re thinking of the relational Codd model with SQL on top of it, it’s not sufficient for doing things like deep learning and so on.

      Martin: Is the same statement true about going from something architected for AI and ML and then having it support more of a traditional analyst relational model?

      Ali: So, interestingly, I think the answer is no, because there is now a widespread data science API that has emerged as the lingua franca for the data scientists: data frames.

      A data frame essentially is a way where you can take your data and turn it into tables and start doing queries on it. That sounds a lot like SQL, but it’s not, because it’s actually built with programing language support so you can do that in programming languages, like Python or R, which enables you to do data science.

      So, now your data is in tables, and it turns out you can now also build SQL on top of data frames. You can get a marriage between the world of data science and machine learning and the world of BI and data analytics, using data frames.

      Martin: I get what you’re saying about the data warehouse, but there’s a lot more than just the data sitting in the data warehouse. You still have this entire world of data and SQL and ETL. Is there a dissonance there or do they stay two worlds? What happens?

      Ali: Every enterprise we talk to, they have the majority of their data in the data lake today, and a subset of it goes into the data warehouse.

      There’s a two-step ETL that they do. The first ETL step is getting into the data lake, and then there’s a second ETL step that they use to move it to the warehouse. So, organizations are paying a hefty price for this architectural redundancy.

      But the question is: do you really need two copies of it? And do you really have to maintain those two copies and keep them in sync? Are you going to have a world in which you have all your data in the data lake and then you do your machine learning and data science on it, and then subsets of it move again into a data warehouse, where you clean it up and put it in that structured form so you can do SQL and BI, or can we do it all in one place?

      Martin: Let’s actually ask that specific question. Because even though the AI-ML is a large market with a lot of value, there’s a ton of existing workflow around BI.

      You’ve got all the dashboarding and tools that are based on SQL for data warehouses, but then you also have folks that want to interact with the data very quickly and will use something like ClickHouse or Druid in order to do that in OLAP. OLAP stands for online analytical processing and is effectively a fast interface that supports fast queries. Then you’ve got more traditional batch processing, which normally folks have thought about Spark. What you’re saying is that you can combine all of these things in the same data lake, including OLAP query loads?

      Ali: Yes, I actually think you can get all the way there. The data lakes are a broad source. Big, large, cheap storage, but kind of data swamps.

      It turns out there are some recent technological breakthroughs that show you how you can basically turn them into a structured relational storage system. The way you do that is you build transactionality into these data lakes.

      Once you have that, you can now start adding things, like schemas, on top of them. Once you add schemas on top of them, you can add quality metrics. And once you have that, you can start reasoning about your data as structured data in tables instead of data that’s just files.

      Martin: I get putting structure on top of a blob store, but you still need a query later, right? Building a query engine that’s super-fast that can respond to analytical queries, there’s entire companies that do that.

      Ali: Yeah, so it turns out there’s two APIs you need. One is the data frame API. That’ll enable all the data science and machine learning. Then you can build a SQL layer on top of it, and there’s nothing that really gets in the way of making this as performant as the state-of-the-art, fastest MPP engines out there. You can apply the same tricks now because you’re actually dealing with structured data.

      The real use case for streaming

      Martin: It feels like, especially in data, there’s always kind of the trend du jour that everybody’s excited about, but they’re not ever really sure if the market’s real or not. People have been saying this a lot for real-time and streaming use cases.

      It’s very clear that people want to process data at different times and speeds. Batch, we know, is a very large market, where you’ve got a bunch of data, you want to do a whole bunch of processing, and then it’s stored somewhere else and you do some queries.

      More and more people are talking about streaming analytics, where as a stream comes in you do the queries before it hits disc.

      I sit in pitches basically as a full-time job, and a lot of the things motivating the streaming use case seem a little a contrived.

      Ali: There’s the latency and the speed and how fast you can get this stuff. That’s one side of the equation, and that’s what everybody focuses on.

      Oftentimes when we ask the business leader, “Hey, so what kind of latency would be okay with you?” They’ll say, “We want it to be superfast like every 5 minutes, every 10 minutes.” And you can accomplish that with batch systems.

      Then when you dig into, “wouldn’t you want it to be even faster?” It turns out that streaming systems, the weakest link will dictate the latency. There’ll be some upstream process that has nothing to do with the system that you’re putting in place. And if that upstream link, if that one place where you’re loading the data in or something, if that’s coming in every half an hour, then it doesn’t matter how fast the rest is.

      I think the actual latency, this obsession with, “We need it in less than 5 milliseconds.” For most use cases, you don’t have that.

      There’s another side of the equation, which people don’t focus on because it’s harder to understand or explain, but it might be the biggest benefit out of these streaming systems, which is, it takes care of all the data operations for you.

      If you don’t have a real-time streaming system, you have to deal with things like, okay, so data arrives every day. I’m going to take it in here. I’m going to add it over there. Well, how do I reconcile? What if some of that data is late? I need to join two tables, but that table is not here. So, maybe I’ll wait a little bit, and I’ll rerun it again. And then maybe once a week, I rerun the whole thing from scratch just to make sure everything is consistent.

      In some sense, all the ETL that people are doing today and all the data processing that they’re doing today could be simplified if you actually turn it into a streaming case, because the streaming engines take care of the operationalization for you. You don’t have to worry anymore: “did this data arrive late? Are we still waiting on it? Is the thing consistent?” They’ll take care of all of that.

      Martin: You think ultimately a large part of this becomes stream processing?

      Ali: What I’m saying, provocatively, is that in some sense all of the batch data that’s out there is a potential use case for streaming.

      I think that stream processing systems have been too complicated to use, but actually under the hood they take care of a lot of data ops that people are doing manually today.

      Ali’s end-to-end architecture

      Martin: I would love to talk through what you think a modern data stack looks like. We talked to a whole bunch of folks, and it seems there’s a best practices stack forming, but very, very few people know what it looks like.

      Let’s say you get hired, Ali. You have a new job, VP of Data, and you were to build a data infrastructure that does both analytics and AI-ML, what product category — not specific products, but product categories — would you use end to end?

      Ali: If I get hired into a big company, I’ll spend the next five years fighting political battles on who owns which part of the stack, and which technology I would need to get rid of. There’s a lot of org chart, and human, and process problems, but let’s say, I get in there and they say, he gets to have it his way.

      Martin: He’s got all the juice, that’s right.

      Ali: Obviously, trying to do something on-prem makes absolutely no sense at this point. And when you’re building that cloud-native architecture, don’t try to replicate what you had in the past on-prem. Don’t think of it as big clusters that are going to be shared by users.

      One big change that happens in the cloud that on-prem vendors don’t think of often is that the networks in the cloud are invisible. Any two machines can communicate at full speed, and it can also communicate to the storage system, to the data lake, at full speed. This was not the case on-prem, and things like Hadoop and so on, they had to optimize where you put the data and the computation had to be close to the data.

      So, you move it into the cloud. Typically, you have data flowing in from some of your systems. Depending on what kind of business you’re in, you have IT devices or you have something from your web apps. Sometimes it goes to streaming queuing systems, like Kafka. And from there, it lands into the data lake.

      Martin: Into the data lake. So you’re saying the data goes directly into your data lake.

      Ali: That’s the first landing place. If you don’t do that, you’re actually going to go back a decade or two in the evolution. Because if you don’t put it into the data lake, then you have to immediately decide what schema you’re going to have. And that’s hard to get rights from the beginning. The good news with data lakes is you don’t have to decide the schema. Just dump it there.

      Step number two, you need to build a structural transactional layer on top of it, so that you can actually make sense of it. There’s three or four of those technologies that appeared roughly at the same time, and they all enable you to take your data lake and turn it into a lakehouse.

      Step number three. You need some kind of interactive data science environment where you can start interactively working on your data and getting insights from it.

      Typically, people have Notebooks-based solutions, where they can iterate with Notebooks. They use things like Spark under the hood, and they’re interactively processing their data and getting insights from it.

      And that’s really important because a lot of data science in organizations ends up not being advanced machine learning. It ends up being, okay, so we have this data coming in from our products or from our devices or whatever it is. We have to massage it, get it in a good form, and we need to get some basic insights out of it.

      If you want to get into the predictive game, you need a machine learning platform. There are now these machine learning platforms that are emerging, many of them are proprietary, inside the companies. You can read about them, but you can’t get your hands on one.

      Martin: And this is for operational ML?

      Ali: This actually goes from training the ML model, so actually featurizing it, creating a model that can do the predictions, tracking the results, making sure that you can make them reproducible and reasoning about them to moving it into production, which is the hardest part. Moving it into production where you can actually serve it inside products. That’s the job of the machine learning.

      Martin: And the people that use the machine learning platform in your world are the data scientists, the data engineers, or both?

      Ali: It’s different organizations, today, unfortunately. The serving part, the production part sometimes is owned by IT, and the creating of the models happens by data scientists that sit in the line of business.

      And there is friction in those organizations, because IT operates at a different wavelength from the data scientists, but the machine learning platform needs to span both. If it doesn’t, you’re not going to get the full value out of the machine learning work that you’re doing.

      Martin: Can you talk a little bit about where the data pipeline and DAG tools fit in in all this?

      Ali: That would be the first step of this. I talked about training immediately. But the hardest part really is to take that data that’s now sitting in the data lake and build the pipelines that featurize it and get it in the right shape and form so that you can start doing machine learning on it. So, that’s step number one. Then, after that, you start training the models.

      To orchestrate that automatically and make that workflow just happen, you need software that does that, so that’s definitely the first mile in the ML platform.

      Martin: And if I want to take my traditional BI dashboard and attach it to this, where does that attach?

      Ali: That’s the last mile. BI itself typically uses something like JDBC/ODBC. To make that really fast and snappy and work on top of the data, you need some capability that makes that possible.

      In the past, your only option has been to put it in a data warehouse, and then attach your BI tool to it. I’m claiming that with the lakehouse pattern that we’re seeing, and with some of those technological breakthroughs I mentioned, you could connect your BI tool directly now on that data lake.

      Martin: To where? To the transactional layer that’s built on top of it?

      Ali: Yep, if you have something like Delta Lake or if you have something like Iceberg or Hive ACID, you could connect it to those directly.

      Martin: If you didn’t have any legacy technology, it seems like doing a data lake makes a lot of sense. Is there a simple migration path to this?

      Ali: I think it’s harder in the West. In Asia, it’s easier because there’s not lots of legacy. It’s harder in the West because the enterprises have 40 years of technology that they’ve bought and installed app data in and configured. They need to make that work with what ywe’re talking about.

      Whereas if you’re building it from a clean slate, you can actually get it right more easily.

      Martin: Are you actually seeing more usage of data lakes for companies that aren’t encumbered by legacy?

      Ali: The companies that are really succeeding with this stuff… take an Uber. They’re doing predictions, and the predictions are a competitive advantage. You press a button, and within a second, it tells you what the price of the ride is. It basically simulated the ride. It knows what that meter is going to tell you after an hour ride with traffic conditions and everything. It gives you exactly the right price — can’t overprice, can’t underprice. It matches supply and demand of drivers with surge pricing. It can even put people in the same car to lower the cost.

      All of these are machine learning use cases, and those stacks, these are all companies that are 10 years old. They didn’t exist. They don’t have lots of legacy data warehouses and legacy systems. They built it custom for this use case, and it’s a huge competitive advantage.

      Where to from here?

      Martin: Is this the durable stack that lasts for the next decade, or is this converging on something that looks a little bit different than you can articulate from here?

      Ali: I can’t predict the future, but I’ll tell you a few ingredients of it that just make sense long-term.

      If I’m an enterprise and I’m sitting there as a CIO or someone that’s picking the data strategy, I would make sure that whatever I’m building is multi-cloud. There’s a lot of innovation happening between the different cloud vendors. They have deep pockets, and there’s sort of an arms race there, so make sure that you have something that’s multi-cloud.

      The second thing I would do is, as much as possible, try to base it on open standards and open source technology if possible. That gives you the biggest flexibility that, if the space again changes, you can move things. Otherwise, you find yourself locked into a technology stack the way you were locked in to technologies from the ’80s and ’90s and 2000s.

      Storing all your data, dumping it first in raw format into a data lake, is something that’s going to remain because there’s so much data that’s being collected. You don’t have time to figure out exact perfect schemas for it and what we’re going to do with it. So, either we dump it somewhere, or we throw it away, and no one wants to be that employee that threw away the data, especially when it’s so cheap to store it.

      And the third thing I would do is I would make sure that the stack that you’re building, the way you’re laying it out, has machine learning and data science as first-class citizen. Machine learning platforms didn’t exist 15 years ago, so that probably will change quite a bit. I think the exact shape of the machine learning platform I don’t think will look exactly the way it is today.

      But many of the ingredients are right.

      Martin: Perfect. Thank you so much. I don’t know if we’re on a race to see who speaks faster, but I think you win.

      Ali: Thank you for having me.

      • Ali Ghodsi

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      Designing for, Marketing to, and Partnering with Gen Z

      Tiffany Zhong and Connie Chan

      Gen Z—those born between 1995 and 2010—now makes up 35 percent of the population and represent $143 billion dollars in spending power. This episode is all about how brands can better understand, collaborate with, and resonate with this hugely influential segment of consumers. 

      Our guest, Tiffany Zhong, is the 23-year-old CEO of Zebra IQ, a company that helps brands interpret the wants of Gen Z consumers and helps Gen Z creators turn their content into businesses. In its recent Gen Z Trends Report, her company highlights important cultural trends and Gen Z behaviors based on a trove of proprietary research. In this conversation, Tiffany and a16z general partner Connie Chan discuss the key differences between Gen Z and millennials, the growing power of short-form video on platforms like TikTok and YouTube, our changing perception of luxury, and how Gen Z is shifting the paradigm around money, education, and work.

      The pair breaks down how brands can partner with Gen Z influencers in a way that’s compelling, not cringeworthy, and why when it comes to memes and the art of emoji, you’re probably doing it wrong.

      Show Notes

      • The importance of marketing to Gen Z, their unique characteristics [1:22], and the platforms influencers use most [3:05]
      • The rise of TikTok and text vs. video [6:20]
      • How Gen Z creates different personas for different platforms [9:22]
      • Tips for finding the right influencer [11:31], and a discussion of how many influencers have become company founders [14:16]
      • Common mistakes when marketing to Gen Z [17:20] and the importance of corporate branding [20:21]
      • Gen Z’s view of money, work, and higher education [22:06], as well as shopping and luxury [25:01]
      • How Gen Z finds friendships and tribes [28:44] and the potential of texting [31:04]

      Transcript

      Gen Z basics

      Tiffany Zhong: If you don’t have the youth using your product or talking about your product or sharing your product, I hate to break it to you: you’re irrelevant. And so that’s why every single company that is targeting consumers needs to care about Gen Zs, whether you’re a Fortune 500 company or whether you’re a startup.

      Connie Chan: Are there perceptions that Gen Z has around millennials?

      Tiffany: Gen Z considers anyone who is not really speaking their language or not understanding their trend, a boomer. It doesn’t matter if you’re a millennial, it doesn’t matter if you’re a Gen X, it doesn’t matter if you’re a boomer, Gen Zs are going to call you boomers anyway.

      Connie: Do you think, though, given how personalized Gen Z’s preferences are, that there is a definitive “this is cool,” “this is not cool”?

      Tiffany: It changes weekly. So you have to keep up if you want to understand what’s cool or not.

      Connie: That’s really hard on the wardrobe, man. [laughs] Would you say that the difference between Gen Z and millennials is a much bigger gap than between millennials and older generations?

      Tiffany: I would say so, because Gen Z is the first generation that’s mobile-first and mobile-native.

      Connie: I totally agree. Millennials will say we’re mobile first, but there’s a lot of stuff that we still feel much more comfortable going to a computer to do. Big ticket purchases, we still feel like we’re safer on the browser, for some reason.

      Tiffany: Whereas Gen Zs do everything from their phone. We’re used to that. We’re used to buying things from our phone, signing documents from our phone. For better or for worse.

      Connie: Requiring much more instant gratification, I’d say. Even the YouTube videos now feel too long to me, if the first minute is the person apologizing and trying to be politically correct. They just need to get to the point, or the ROI has to be real.

      Tiffany: Yeah, if it’s good content, if it’s entertaining content. If not, then boom, we’re out. And you’ve lost us.

      Connie: When people say “Okay, I want to go be an influencer now.” Before, for millennials, you became a YouTube star. Now is it more desirable to be a TikTok influencer versus a YouTube influencer? Certain [platforms] are clearly easier to go viral on.

      Tiffany: Twenty-nine percent of youth in America want to become vloggers or YouTubers, versus 23 percent want to become professional athletes. So more people want to become YouTubers than athletes, which is a massive shift.

      On the platforms that Gen Z wants to be an influencer on, TikTok seems the easiest for people because we’ve obviously seen Charli D’Amelio becoming one of the biggest influencers in under a year.

      Connie: Totally. The conversations around the actual TikToks right now are living on other platforms, but it’s super valuable.

      Tiffany: Exactly. TikTok stars are all spending time on YouTube now. It’s a natural growth phase. So when you say: are TikTok stars the new YouTube stars? There’s a whole correlation there, in the sense that if you’re big on TikTok, the way you can really continuously build an audience that is sustainable is on YouTube.

      Connie: But many have not been able to be as successful on YouTube. It’s more likely they will not be successful, actually.

      Tiffany: Because they haven’t really adapted on how to make long form content. They’re used to making 60-second videos, which doesn’t translate well to YouTube. On YouTube you want to be making 10 minute videos, because that’s how you monetize. There is no easy virality factor. You just have to be really good at distribution and marketing, honestly.

      Connie: Especially on TikTok, there’s so much remix meme culture. It’s not necessarily 100 percent original. And maybe that’s part of why it’s so hard to translate [TikTok content] to YouTube: for the most part you have to come up with something completely original.

      Tiffany: People who are really, really good storytellers will be able to do so across different mediums, whether it’s TikTok, whether it’s YouTube, whether it’s Instagram, whether it’s Twitch.

      Connie: I think for short-form video, especially the stuff on TikTok, it’s about: what’s the punch line? What’s the actual point of the video that makes it interesting? And that’s why it so democratizes video creation. You don’t need a ring light to be a good TikTok creator. You literally just need your phone.

      Tiffany: Yeah, on TikTok, you get 30 seconds, you can record it with your phone, you can have whatever quality of video, and as long as you have a good storyline people will watch it and people will share it. Or if you’re adding value to the viewer’s life.

      Connie: Yeah, short videos are not just jokes. I mean, I cringe whenever people say TikTok is a bunch of people dancing to music, because I’m like: you clearly have not used this thing. [laughs] There’s educational stuff on it. There’s financial advice on TikTok. There is stuff that teaches you how to cook. So short video is a really powerful format, I think. And it’s basically getting rid of the fluff that you don’t need and delivering maximum value per second—literally, per second because you can lose the person after three or four seconds if it’s not good enough.

      TikTok and the preference for video

      Tiffany: To your point, people think TikTok is just a fun lip-synching app or dancing app. And it’s not. It’s a place where you can learn anything you ever wanted to learn, whether it’s about cars, whether it’s how to take photos, how to model. I’ve watched so many TikTok videos about videography tips and iPhone tricks, all sorts of stuff. It’s just endless amounts of short-form education.

      Connie: I think that phrase has never been used to describe TikTok: short-form education. I’m curious on your thoughts about on the kind of content that historically people would argue works better in text. How does Gen Z react to, you know, that super thoughtful op-ed on the New York Times, or product reviews—things that you can actually read much faster than you can watch?

      Tiffany: Gen Zs prefer video over text for like 99 percent of things.

      Connie: How do you balance that with efficiency, though, where you can actually read some of these things much faster than you can watch for some of these things?

      Tiffany: True. But not only do we want to be able to consume the content in a reasonable amount of time, we also want to be entertained at the same time, which is why video is such a huge format for Gen Z. Text is less relevant because there are less emotions. You can’t see someone talking. Sixty-five percent of Gen Z prefers FaceTime to any other form of communication to keep in touch with friends.

      Connie: And people don’t realize that when Gen Z is doing a FaceTime phone call, it’s not like they have to hold the phone the whole time.

      Tiffany: Oh, yeah. They might be video chatting their friend or their parents while simultaneously doing like three other things. Multitasking is what we were born into because of smartphones. We’re used to switching between tabs quickly, switching between apps quickly.

      Connie: I definitely see different communication behaviors across different generations. One thing I think people don’t realize is just how many young folks have multiple Instagram accounts, for example, or multiple Twitter accounts, because they have to show different aspects of their personality and segment parts of their lives.

      Tiffany: Every person has dual personalities. You have a personality that you bring to work, you have a personality that you bring to your friends, you have a personality that you bring to your family.

      Connie: I have many more than that, but yes.

      Tiffany: And, and so that’s how Gen Zs have started to establish themselves. They want to be able to be super fluid and switch across these different identities. This finsta—fake Insta account—which is really just for personal friends, this one’s for this set of friends, or this one’s for this set of interest-based friends. This one’s for this community. That’s how these finstas start being created.

      Connie: But it’s more like on TikTok, they can be a different version of themselves. On Instagram, they might still keep that polished version of themselves. You have different personas on different spectrums of that authenticity scale. And on different [platforms], you’re going to reveal more information or less information about yourself, too. Some you’ll reveal your actual name, where you live. Some it’s all random usernames, on purpose. There’s more control over what people can see and how they would use it.

      Platforms and personas

      Tiffany: Gen Z is definitely very smart about the perception that they put out there across different social media networks. Gen Zs are brand strategists from age 10. They learn: okay, my Instagram needs to be like this, my YouTube needs to be like this, my TikTok needs to be like this, my Twitter needs to be like this. It’s so different than how millennials and Gen X perceive content.

      Connie: I definitely think the way that millennials grew up on social was to put our best foot forward. You always wanted to make sure the photos that you were posting reflected well on you, or you would untag yourself on the Facebook photo so it wouldn’t be linked back to your profile. We used to all do that. And just think about all the filters that we use on our photos, all the photo apps. But I do feel like there is this change swinging back to: don’t put a filter on everything. Or: it doesn’t have to be in the most flattering angle. But it’s not necessarily that they will do that across all social media.

      Tiffany: Your main Instagram, you still care about your follower count, you still care about your likes, you still care about your comments. For finstas, it doesn’t matter as much. If you get one “like,” it doesn’t matter. Because it’s really just where you can be your real self.

      Connie: Something I’ve noticed on Gen Z and TikTok is there’s less of a fear of being on video; there’s less of a fear around creating in general.

      Tiffany: Totally. TikTok has made people really comfortable with being themselves.

      Connie: Showing the no filter life.

      Tiffany: Yeah. Because the weirder you are, the more chances you will go viral. The YouTuber Emma Chamberlain is one of the fastest growing Gen Z influencers. She has one of the highest engagement rates across young influencers. Now, her content is all very authentic, she’s very much herself. She mixes in that very relatable aspect with the very aspirational. And I think the best influencers are able to be both aspirational and relatable.

      That’s why raw photos, raw videos are actually bridging the connection between influencer, creator, and fans. A really polished version of yourself doesn’t seem very attainable. When you’re a fan sitting at home, you want to feel like you could be that influencer too, someday.

      All about influencers

      Connie: How do you find the right influencer to work with? Historically, people just look like at, okay, how big is your following? What should they be thinking?

      Tiffany: You look at the type of content they’re posting. Is that similar to the type of content you post on your social media? Is it on brand? Are they in your niche? Are they already talking about products or your space in general? And then there’s the checking if they have real fans and authentic fans by looking at their engagement rate.

      SocialBlade is a really simple website that lets you look at any accounts, any pages on social media across YouTube, Twitch, Twitter, TikTok, Instagram and see how fast someone is growing, see how many followers they got yesterday, how many followers they got 7 days ago, 30 days ago; how many followers they lost, as well. And so that’s a really authentic way to go and track how fast an account is really growing. You can see their relevance through growth and their engagement through that.

      And I think you will see that a lot of these smaller influencers actually have really, really high engagement rates because they have more time to spend, so their fans reciprocate. Recently, an influencer called Bella Poarch started becoming super relevant with her head bobbing TikTok videos. She’s really blown up. Now, you could have spotted that a couple months ago if you just looked at SocialBlade and watched how she literally grew exponentially.

      Connie: So for Gen Z, how would you balance brands choosing influencers versus traditional celebrities—people from movies, TV shows? You’re laughing—I feel like you have an answer that’s probably contrarian to what a lot of marketers believe today.

      Tiffany: Celebrities still give you that legitimacy factor to a certain extent.

      But it better be extremely on-brand to be working with this very specific celebrity that you choose. Not because of their fame, but because maybe they’ve talked about your brand already, or they drink your brand, or wear your brand, or use your brand, or eat at your restaurant, whatever it is. There has to be something like that there. Do not pay a celebrity a million dollars to promote a brand that they don’t give a shit about. I’ve seen many brands that have just burned money on celebrities.

      Influencers are good for more authentic collaborations that are closer to home for the fan. Celebrities are not relatable. And so I think there’s a good way to mix in both celebrities, massive influencers, and also micro influencers, if you really want to be strategic in how you utilize your money.

      Connie: So celebrities and superstars still exist. It’s just the ones that have lasting power are the ones that feel like they’re your friend and have some level of being relatable.

      Tiffany: Aspirational and relatable. Gotta be both.

      Connie: On the influencer side, do you believe that the lifespan of someone’s popularity has also shortened in length? Where previously you might have a celebrity or an influencer that you love and you follow for like 10, 20 years, do you feel this new generation is going through them quicker? How do you think about the lifespan of content, movies, TV, influencers themselves?

      Tiffany: Your shelf life can actually be extremely long if you think about it from a very strategic standpoint of brand building as an influencer. Now, a lot of influencers aren’t really thinking about a 10-year lifespan. They’re thinking “how can I make as much money in the next year as possible?” And I think that is a huge problem because they aren’t treating themselves like they are their own media companies, they aren’t treating themselves like they are a company and they are the CEO.

      The people who have had really long shelf lives are people who have adapted with their audience, people who listen to their audience, the people who engage with their audience and make their fans feel like they are being heard. As your audience grows older, your content adapts, as well. You grow older, your content matures a little, and your fans grow older, as well.

      It’s more important to have longer term customer retention and lifetime value than customer growth. It is more important to have 1,000 super-fans than 10 million fans who will never buy anything from you.

      Connie: As these short video platforms potentially go into commerce, what are your thoughts around creators and influencers making merchandise themselves—and becoming stores, really?

      Tiffany: I think creators are starting the new billion-dollar commerce brands and the new billion-dollar media companies. We’re seeing that with people building tech companies, [like] David Dobrik. He built an app, raised venture financing for it. His merch brand is doing incredibly well.

      Connie: But you would also have to say he’s one of the top YouTubers. He’s not indicative of most influencers.

      Tiffany: He’s not, but he is a really good role model for a lot of creators and what they can do. Any creator that has a really strong fan base can establish their own commerce brands. They are a media brand already because they are creating content. Now how can they parlay that into something that is relevant to their audience?

      Connie: And it’s also very dependent on how the platforms allow you to monetize, either through ecommerce capabilities, or more gifting, more memberships, you name it. The platforms so far in the Western world have not done very much.

      Tiffany: A SKU might be a phone call, as SKU might be shoes that they have designed, all sorts of different product lines that they’re coming out with. The YouTuber that I mentioned, Emma Chamberlain—really young, really big audience. She started her own coffee brand and it’s doing really well with Gen Zs.

      Gen Z marketing mistakes

      Connie: What are the big misconceptions or the big mistakes that brands have made when they’re trying to target Gen Z?

      Tiffany: I think the biggest mess ups are when brands randomly jump onto bandwagons or trends without fully understanding where the trend has come from, what the trend means.

      Connie: And how long it can last.

      Tiffany: And how it is relevant to Gen Z. If you don’t speak Gen Z’s language, but you try to without actually spending the time to understand it, you get laughed at and mocked on the internet and turned into a meme, negatively. That is when you become a very cringeworthy brand.

      Connie: I find when brands try to use memes, though…

      Tiffany: It’s cringe.

      Connie: Sometimes they get it wrong; they very often get it wrong. [laughs] So I would say for a brand, if you want to use a meme see what the community comes up with first and then just retweet that kind of stuff. Do not attempt to create your own version of it.

      Tiffany: Reproduce it, and it comes out cringe, it comes out awful. You have to understand the origin of it.

      Connie: How long does the memory of that cringe reaction last?

      Tiffany: Depends on how viral it goes. if it goes really viral and there are press articles about how bad it is, then it may take longer to recover. A recommendation I have for anyone who is trying to understand Gen Z trends per se, is: open TikTok. Don’t just watch the TikToks…

      Connie: Make one.

      Tiffany: Make a TikTok. Read the comments. That is how you can understand Gen Zs really, really quickly. Put in the work and you’ll actually be able to do an awesome campaign. But reading the comments on Gen Z TikTok pages, reading the comments on TikTok gossip pages like TikTok Room on Instagram. Those are all Gen Zs. That means all the comments you read there are posted by Gen Zs. And that means that is how they’re talking. Whether it’s trends they’re talking about, whether it’s slang terms they’re using, whether it’s emojis they’re using to express themselves.

      Connie: So, the art of texting. One of my favorite slides in your deck was actually the emoji dictionary, where it was showing that the traditional happy face is actually not a good thing to send to a Gen Z because it can be [interpreted as] an extremely passive aggressive smile.

      Tiffany: It’s part of our personality to be self-deprecating and to be really honest, but masquerade that honesty in a joke.

      Connie: This is why sarcasm is now so hard to read through text. It seems very easy to misread a text, and now misread the emoji. I did not know that the cowboy emoji, in your opinion, is actually a negative thing too.

      Tiffany: Oh, yeah. It was very surprising for a lot of people. I’ve actually converted a few Gen X friends into Gen Z-style texting. I had to teach a Gen X about how to do text reactions.

      Connie: Text reactions, as far as I know, are okay.

      Tiffany: Texting the thumbs up emoji is like…

      Connie: It’s an acknowledgement.

      Tiffany: …passive aggressive.

      Connie: Oh my gosh. [laughs] So you talk about needing to experiment and being willing to figure out how to talk to this generation, or else you could be laughed at. Do you believe there is a correlation between Gen Z, cancel culture, and an increased fear of speaking incorrectly to this group?

      Tiffany: One hundred percent. I think with this year has come an increased fear of being canceled on social media, especially when many brands acted incorrectly. Brands who had never thought about how they would engage in a political discussion were suddenly forced to do so. And many just didn’t adapt fast enough. Although some did and some were applauded for it by Gen Zs.

      Connie: I’d be curious about your thoughts around the need for companies to be transparent on either how they’re making their money or how much money they’re making.

      Tiffany: Gen Z is very perceptive of the brands that they buy from and that they shop from, and also the places that they want to work at. And so they’re very value driven around human rights, and the environment, and political reform, and education. With that, brands need to figure out what they stand for and live by it.

      Connie: Do you think brands are able to stay out of that discussion and not have a stance?

      Tiffany: Not having a stance is taking a stance. Not having a stance means you don’t care about these things. If you really want to appeal to a wide range of Gen Zs, figure out what your values are and live by it. Talk about it, make that part of your brand. Lean into it and make that part of your whole brand marketing strategy. But don’t jump from value to value really quickly just because it is trendy. Gen Zs will see right through that.

      How Gen Z views money and work

      Connie: One thing that people often talk about is how Gen Z is really good at figuring out how to make money on their own to buy the things they want at a younger age, versus relying on parents. I would love to hear your thoughts on money and work as a category.

      Tiffany: I think there’s going to be a future where work is more project-based. The new American Dream for Gen Z is being able to work wherever we want, whenever we want. Now, Gen Z is the side hustle generation because from a young age we realized that we can hustle to make money online.

      I really break it down into three categories. Freelancing: so, Fiverr, graphic design, etc. Making strategic investments: building a GOAT and Grailed store, building meme pages, selling ads on the meme pages or flipping the meme pages, buying and selling the right kinds of street wear, getting 10X what you paid for. And then the third category is creating content: becoming a fulltime content creator on TikTok, YouTube, Instagram, or Twitch. Gen Zs are realizing that we can make money in all sorts of different ways. And we also don’t have to be tied to one place. We don’t have to be tied to one 9 to 5.

      Connie: And this has to extend to college, too, as people look at college student debt. One big question has been what is higher education going to look like, as people don’t always see the return on investment? There’s no shortage of curriculum.

      Tiffany: I do think that a lot of kids benefit from higher education. I’m not sure if these prestigious colleges are exactly where they should be spending their time and their money, both for opportunity cost, but also debt-wise. You’re spending four years getting a degree that you may or may not end up having your lifetime career in. And you have that debt that you have to pay off for the next 10 years, which is crazy to me. So I think gap years are interesting. All these alternatives: apprenticeships, internships.

      Connie: Do you think that desire to try different things also extends to post-college, potentially more job hopping?

      Tiffany: With Silicon Valley and millennials, specifically. It is common to be in a job for two years and switch to another one.

      I think the gig economy is going to become even more relevant for Gen Zs because it gives Gen Zs the freedom to do whatever they want. So, not having to sit inside an office from 9 a.m. to 5 p.m. every single day, having to request for paid time off. If you work for yourself and if you are in the gig economy, or if you’re flipping shoes, or doing these side hustles, or turning these side hustles into real businesses, then you might have more time to travel or become a content creator. Gen Z really wants to be able to explore different categories, different types of work.

      Connie: Yeah, and they need the freedom to do it. Let’s also talk about shopping. What does luxury even mean now? Streetwear can be very, very expensive, actually. A black baseball hat can be very expensive. What do you think about the future of luxury?

      Tiffany: Yeah. So thrifting’s becoming cool and relevant again. A lot of influencers who are wealthy are thrift shopping and they’re showing their thrift hauls.

      Connie: But that value needed for content—that it better be good value per second—is that also extending to actual purchases?

      Tiffany: Value is important, but so is convenience and so is staying on trend. To a certain extent, Gen Z is voting with their dollars. But there’s always that convenience factor that also comes into play. And so brands like Boohoo and SHEIN are really relevant with Gen Z and really popping off with Gen Z because of the fast fashion nature of it. I get these ads from SHEIN and it’s $10 for like, a sweater. This is crazy. Where’s this even from?

      Connie: But will you ever grow up and then say, okay, I’m now cool paying $600 for that sweater? Not that sweater, but a sweater.

      Tiffany: I think that the future of fashion is going to be a mix. It’s going to be a mix, in my opinion, of people being able to put together things that they buy at the thrift store combined with the latest Off-White shoes, the latest Yeezy shoes.

      Connie: But that’s not that different. Millennials also spent a lot on a purse or a belt. Same thing.

      Tiffany: Right. But you get embarrassed to wear cheap clothes, as a millennial…

      Connie: I’m cool doing it. [laughs]

      Tiffany: …which I think is becoming less of a thing. You don’t get shamed for wearing cheap clothes anymore.

      Connie: I also want to hear your thoughts overall on how shopping behaviors are changing. How do you make shopping fun?

      Tiffany: In regards to shopping, there are more stores doing pop-ups and doing these limited-edition or time-based activations that are really cool, really relevant, to lure Gen Zs to come in, try out the products, and take some cool photos. You’re seeing a lot of direct-to-consumer brands making pop-up stores or even just going brick-and-mortar, more as a marketing expense as opposed to a place to drive sales, which is very interesting. There’s obviously a massive paradigm shift there, when you see these consumer brands that are backed by VC firms spending the capital that they otherwise would have spent on ads to do in-person activations. They realize that having people be able to tangibly see something, touch something, is still just as impactful as being able to order stuff online.

      Connie: So brick and mortar still matters?

      Tiffany: There’s obviously lots of ways to make your brand fun and interactive. For Gen Zs, their “third places” are all digital, which makes sense. It’s Fortnite, it’s Discord, it’s House Party, it’s Twitch, it’s even apps like Squad. Those are the places where Gen Zs are making new friends, hanging out with their old friends. So, gone are the days where all of your best friends have to be within a mile of you. Now you can have a best friend who is 4,000 miles from you and you can still have as intimate a connection as someone who is a block away from you. Only the internet has made this possible for us. Technology has made it a lot easier for us to make friends with people who are also interested in gaming, or fashion, or basketball—really as niche as you want to go.

      Finding friends and tribes

      Connie: When you’re finding friends, how much of it is people who like the same influencers, the same brands? How much of it is interest-based? Has that changed? How do you even go about finding your tribe online?

      Tiffany: Gen Zs are finding their tribe through being able to search in specific hashtags or specific rooms of people who are interested in same things. Going into subreddits, that subreddit leading to a Discord community of people who came from Reddit, to wanting to chat in a room. There are lots of Gen Zs who are tweeting a list of their favorite influencers. And in that tweet it includes, “Hey, if you’re also interested in—insert influencer’s name—DM me and we’ll add you to our group chat.” So, brands, influencers, they all fall under interests now. That is an “interest.” This is the modern-day Facebook pages. You’re co-signing it by buying the merch, by tweeting about it, by making stan pages on Twitter and Instagram.

      Connie: I look at trends that are happening in developing countries, specifically China and Southeast Asia, and they’re very, very mobile-first, too. Even older generations are mobile-first there. And I think that has led to the development of more things like the superapp model, where you have one app that does multiple things. Do you think Gen Z will be more receptive to something like that, versus older generations in the Western world that still seem to prefer one app that does one thing, for now?

      Tiffany: For this generation, we’re optimizing for convenience. So if things are bundled together, that saves us time. I think that’s really important for Gen Z.

      Connie: Fewer taps to do the same thing.

      Tiffany: Yeah. I mean, we’re consumed by so many notifications, so many products every day, so many apps every day, so much content to consume that I do think that there is going to be a massive bundling of things. It’s going to be really tricky to get right. But we’re already seeing a lot of bundling happening for Instagram, including Reels, commerce, shopping, etc., that they would not have done five years ago. Even three years ago they wanted separate apps for each kind of function because that’s how we thought about apps.

      Connie: I want to talk about texting as a potential new channel. More and more, when I’m shopping on a site, I’m getting a text that gives me a discount code if I purchase it right away. Or it tells me when something’s being shipped. You also hear about Gen Z not opening email. Talk about texting as a channel.

      Tiffany: Texting is now becoming a replacement for email. But does that mean that texting is actually becoming less personal of a communication format, now that advertisers and brands are texting us?

      Connie: Yeah, so as that increases, do you see the same issues as email where you’re going to want to filter this stuff out, eventually?

      Tiffany: One hundred percent. Once we start getting bombarded, we’re going to become more selective, or a new medium will become more relevant for us. SMS shouldn’t be our email inbox.

      Connie: Because it’s weird, right? You can do far less on a text than on email. And it’s actually much more invasive.

      Tiffany: I barely even give people my number, let alone companies my number.

      Connie: But then at the same time, when I receive these texts, they definitely work on me. And I do click in, and I sometimes do complete that purchase.

      Tiffany: I guess it’s for the brands that you really, really, really, really love. If they’re texting you, you don’t feel a sense of invasion. Now, you would only feel comfortable with that for very select brands that you’re a huge fan of, where you want to be notified when something has launched. You want to get alerted before it goes out to the public. You can use SMS as a way to facilitate that intimacy between a brand or an influencer and an individual—ideally with two-way communication. And so you’re going to offer some sort of value, whether it’s discounts, whether it’s getting something exclusive that others can’t get or will get later on. So you’re a super-fan of those brands.

      Connie: Thank you so much. It’s been so much fun chatting about Gen Z, and I will be much more self aware now the next time I text you with an emoji.

      • Tiffany Zhong

      • Connie Chan is a general partner at a16z where she invests in consumer tech. She's well-known for her deep knowledge of the Chinese consumer tech landscape and spotting those trends moving from east to west.

      The Present Future of Audio — Talk, Music, Video, Interactivity

      Gustav Söderström, Connie Chan, and Sonal Chokshi

      We’ve already talked a lot about podcasting, both evolution of the industry as well as the form, but where are we going with the future of audio, more broadly? Can we borrow from the present and future of video (e.g., TikTok) to see what’s next in audio (more layers, more interactivity)? Can we borrow from the past of audio (i.e., radio) to see what’s next for audio experiences (more blending of music, talk, podcasting)? Where do all these mediums converge and where do they diverge — when it comes to user experience, product design, recommendations, discovery?

      Gustav Söderström, chief R&D officer (who oversees the product, design, data, and engineering teams) at Spotify — the world’s most popular audio streaming subscription service — joins this episode of the a16z Podcast for a deep dive on all things audio with a16z general partner Connie Chan and editor in chief Sonal Chokshi. They cover the past, present, and future of audio — going high level into the big trends and also dipping down into the trenches — especially given the increased blending of talk/ podcasting, music, more. What are the challenges to designing for different mediums, on both front end and back end (including machine learning and different graphs), when listeners want everything in one place when and where they want it… yet their contexts shift?

      But the conversation more broadly is really more about what happens when we give creators (of all kinds!) tools — not just for expression but for fan engagement and monetization too. We also discuss the themes of super apps and full-stack approaches when it comes to innovating on top of a protocol, as well as how innovation happens in practice: How do mediums — and organizations — evolve, prioritize, “disrupt themselves”? All this and more in this episode.

      Show Notes

      • How audio and podcasting have changed [2:30] and the merging of music and video through TikTok [5:01]
      • The importance of mobile devices [10:21]
      • Augmented audio and interacting with creators [13:04]
      • How Spotify designs products [16:19], and building “super-apps” [17:58]
      • Technical challenges to integrating media types [24:07], how audio is defined as it merges into new forms [26:54], and licensing issues [32:26]
      • Recommendations and discovery algorithms [36:54], challenges platforms face [43:11], and the importance of subscriptions [47:13]
      • Thoughts about the future of audio [53:12]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. And today we are talking about one of my many, but actually probably [my] most favorite topics: the future of audio. Our special guest is Gustav Söderström, the chief R&D officer of Spotify, which is the world’s most popular audio streaming subscription service. As a reminder, none of the following should be taken as investment advice. Please see a16z.com/disclosures for more information.

      Also joining this episode is a16z general partner Connie Chan, who covers consumer, writes a lot about tech trends and product in China and beyond, alternative monetization models, and more. And she and I have actually done a couple of podcasts on podcasting. One, a podcast about podcasting with Nick Quah. And the other, on how we, at a16z, podcast. You can find both of those episodes as well as other resources on the topic at a16z.com/podcasting.

      Note, also, that Spotify actually got into podcasting in 2015. We were actually included as one of their launch partners for that, among select others, and say we’re huge fans of the pod.

      Gustav: We still are, so it’s still true.

      Sonal: Thank you. Anyway, in this episode, we actually go beyond podcasting to talk about the broader category of audio — past, present, and future. So we chat about the parallels and differences in audio and video including referencing an episode I recently did with Eugene Wei on TikTok, which you can also catch in this feed. We discuss the trend of interactivity as well as augmented audio, and where we are right now. What’s possible, what are the challenges? We talk about where podcasting and music converge and diverge, both on user experience and design, as well as technically in machine learning. And, finally, we go deep on recommender systems. The idea of “hearing” like an algorithm and where subscription models come into machine learning.

      But we also talk throughout this episode about the trade-offs of full-stack approaches, regardless of what kind of company you are, and the topic of super apps as well. And we’re also really talking about how innovation happens in practice. Whether it’s having an opinionated point of view about the future, or listening to users, disrupting oneself — and how to change an organization, and much more.

      But we begin, however, with a super quick debate on how much things have or haven’t changed in the podcasting world. At least, since we did our last podcasting episode over a year and a half ago.

      The current state of audio

      Connie: I actually personally think that audio hasn’t changed that much yet. A lot of things are still — I don’t know if broken is the right word. But just — problems that are not solved yet. Discovery is still difficult, search is still difficult. It’s really like a one-way listening experience. You aren’t interacting with other listeners, you aren’t interacting with the creators. Creators still have to rely on very old business models for monetization that ultimately don’t work for a lot of long-tail creators. A lot of those big problems still exist. But I do have this optimistic feel that we’re on the cusp of change that’s going to come to the broader audio market.

      Sonal: You’re right, those things actually haven’t changed very much. I was thinking of the fact that the content landscape in podcasting has super exploded. In the last year, two years alone, Spotify itself has led a number of content acquisitions, which is such an interesting evolution.

      Gustav: Yes. It’s both very much the same, but very much more of the same, right? So, like, the forklifting of your time into your AirPods, that just keeps increasing.

      Sonal: Right.

      Gustav: There’ve certainly been shifts in listening behavior due to COVID. A lot of listening was in the car, that shifted to speakers in the home — so, overall, there’s much more listening. And to your point, certainly, we’ve invested aggressively in content and exclusives. The creator side of this landscape has changed in a direction that we wanted to change.

      But I would also agree that we’re on the cusp on the consumer experience. What’s so interesting about audio is, it feels like you have this cheat sheet, which is what happened in video. We just haven’t done monetization in a 21st century way yet. We have no interactivity. You can really just look at the other media industries and see what’s missing, in a sense.

      Sonal: So, Edison Research, which publishes a lot of the leading work and studying podcasting behavior — they argued a few things last year. That one of the major inflection points in podcasting, interestingly, came through Spotify because of the streaming. And that brought in, kind of, a new generation of users. Two, the other argument they made. And this is, of course, pre a lot of the content acquisitions — is that for a new generation, the medium of audio is really not that different than video. That, in fact, for a lot of people, their default podcast player is often a video app, or just turning off the visuals and listening. And so, I’m curious, for your guys’ thoughts on where audio and video — which is another big trend — do and don’t intersect? Both from a trend perspective, and a product development perspective, and then we can dig in deeper on other aspects.

      Audio vs. visual

      Connie: I mean, video is really just the combination of using your ears and your eyes. It’s the audio plus the visual. Which means the stakes are actually higher for audio, because I can’t have, like, a 20-second gap of silence in a podcast and expect you to be okay with it. But in a video, you can go quiet and there might just be some visual distraction, and you don’t have to be “on” as much every second. And so it’s still a different medium. But I do think that the stakes in audio are higher.

      Gustav: So I think that when you talk about audio, it’s different things, depending on the type of audio, actually. So you have, kind of, foreground audio, which is more similar to video. It is the main activity you’re doing. You’re really concentrating. It requires most of your attention. Then you have background audio. Like, you’re listening to music, and you’re actually paying attention to something completely different. You’re working out, or you’re studying or something, right? So there are these different modes of audio that don’t really exist in video. Video is mostly all your attention, or you’re doing something else, right? 

      This is also the benefit of audio. That’s why it’s so much engagement, because you have both foreground moments and background moments. But even in the foreground moments, when you’re paying full attention, you can still do other things. You can drive, you can do dishes, you can walk around the house, right? So, it is this other mode that video doesn’t cover. That’s why we think it is almost as much engagement as foreground video, but it’s not nearly valued the same yet. And that’s not because it’s less valuable — we think that’s because it’s undervalued.

      And you can think about it the other way as well. You have some video that actually works quite well as audio, that you can background, that you watch every now and then. Joe Rogan, for example — it certainly has video, right? And that actually does help the user experience. But it is what we call backgroundable video or foregroundable audio, if you want to call it that.

      Sonal: I just wanted to comment, Gustav, on your point about the modes. That’s a phrase that I use when I think about describing people’s behaviors. And I actually describe it less as foreground and background, and more as passive versus active mode. And so, I really believe strongly that audio has different modes. Sometimes you’re just in “hanging out in chill” mode, sometimes I’m in passive mode, which means I just want to listen to other people. Other times I’m in active mode, which means I want to talk, or super active mode which means I want to lead a discussion. So I just think it’s really interesting to think in terms of modes.

      I’d love to hear your initial thoughts on just the mediums differences between audio and video. What do you make of the differences and similarities between TikTok, and what we can and can’t learn from TikTok when it comes to product in audio? Do you guys have any thoughts on that? I mean, Connie, you’ve written so many posts about TikTok since very early on.

      Connie: Yeah. Like, TikTok is an extreme example. If you don’t look at the screen and you just listen, none of the videos make sense. You’ll miss the punchline, like, the whole video.

      Sonal: Yeah.

      Gustav: Exactly.

      Connie: Value prop is also within the visual for TikTok.

      Gustav: So, I think there are at least two similarities. What they do really well is —  they take, to Connie’s point, commodity music — that if you just listen to it in the background, you miss the whole point. But then they let their users uniquify that commodity music, right, by adding uniqueness to it with their video.

      Sonal: I think you just made up a word, by the way, uniquify.

      Gustav: Yeah.

      Sonal: Keep going.

      Gustav: And I think that’s a great pattern, right? You have something that is commodity. You can use your user base to turn that into something that is non-commodity. It’s this engine that takes these clips and creates unique content around it. So I think that’s a really interesting pattern that you could probably copy to other businesses that has commodity content. Let your audience do something with it to make it unique.

      The other analogy that I see to audio is specifically music. If you think about Eugene Wei’s post on seeing like an algorithm. What he said was that the medium itself is built to be understood by an algorithm. You’re presented with one item at a time, you either consume, or you swipe. So it’s built for the algorithm to understand what you’re paying attention to versus, for example, a scrolling feed, where the algorithm has no idea which item your eyes are actually looking at.

      Sonal: Right, isolating the specific variables so that the product developer knows what is working or not working, essentially, for the user.

      Gustav: Exactly. And if you think about music, actually, it’s the exact same thing. You present one audio track at a time. You either listen to it or you skip. So, in that sense, you can say it’s a similar sort of UI, but in audio.

      Connie: The tricky part is actually just the length of the song versus the length of the TikTok video. Because you get to a very quick decision if you like that TikTok video or not — literally within, like, two, three seconds. For a song, as many of you know. Like, the first couple of seconds of a song doesn’t sound anything like the chorus or the ending, so you just have to go further into the song before you really gauge if someone truly likes it or not. But to me, that’s the only difference.

      Gustav: Yeah. And TikTok, you have more evaluations per minute because they’re shorter clips. But it’s also more direct. But it is interesting that you mentioned this, because this is what is happening in the label industry. It is super clear that intro matters more and more, so you do have the TikTok effect in music. You know, songs used to start slow, they don’t anymore because people skip within the first 10 seconds.

      Sonal: Oh, that’s so fascinating. So the TikTok effect — where people are now creating different kind of music.

      Gustav: I would say one more thing on TikTok. So, while there are some similarities between evaluating audio one track at a time, and evaluating video one track at a time, there’s a big difference which is — TikTok has your full attention. If you’re at full screen and you’re paying full attention, then it’s a pretty good signal. But if you’re washing dishes and listening on a speaker, you get very poor signal. So it depends on the context and you have to take that into account when you look at the signal.

      Sonal: I’d love to probe briefly on this part. Which is, you both have talked a lot. Connie, you, in particular, have written so much about how mobile is literally the thing that made a lot of China’s apps work the way they do, because everything was mobile first. And we talked about mobile leapfrogging in our posts from what now, five years ago?

      Connie: Right, right.

      Sonal: Wow, that’s been a long time. So, where does that come in when you think about innovation in audio? And then, Gustav, I’d love your thoughts on this as well. Because when you said that in the pandemic, a lot of the listening behavior has shifted to home speakers, I’m curious how that changes your views, given [the] initially mobile default interface?

      Connie: So, if I just break down what a phone is and the different components of it. Like, you have the touch screen, which means whatever you’re doing on the phone, you can have more interactivity, ideally. But you also have camera and GPS. And, you know, the camera is the unlock for TikTok, and the microphone could be the unlock for a bunch of audio platforms. Because, now it means that I don’t just have to be listening. I’m not just leveraging the speaker on the phone, but I’m leveraging the microphone and I’m giving back. The microphone, in particular, for audio and video, I think is dramatic.

      Gustav: Yeah. That is one of the sensors that is super interesting and under-leveraged for audio, I would say. So, one of the benefits of being a streaming service is that we understand the consumption situation. We understand if you’re listening on a speaker but putting on an Apple Watch or a phone — we understand if you’re in your car, for example, because the phone is connected and so forth. So we actually think that’s a very important signal, and we try to think of them as, kind of, different jobs to be done. And what we want to try to understand is the situation that you’re in. And it’s obviously a combination of your play history, your time, and your taste. But a device is actually a really good signal.

      So there are two levels. One is the UI and the hardware that you can leverage. And that changes when you go from a phone to a connected speaker, for example. You have much less control. You actually still do have a feedback channel, in terms of a microphone, as Connie mentioned. But you have less UI, right? So we’re thinking about multimodal consumption quite a lot, where you have some devices that are really good for input on your body, but they’re not that good for output — you actually want the sound in your speakers. That’s why we built this remote-control protocol so that you don’t have to interact in the same place that you’re listening —  you can interact on one device and so forth.

      The other way to think about it is on the content level. So one of the things that happened during COVID, when a lot of consumption shifted from the car to the home, was that we have this very successful playlist called The Daily Drive, where we mix music and talk — and create, literally, your daily drive. Now people stopped driving, right? So then we tried to pivot and we create [the] same job to be done, but not while driving — it’s different. So these are the two levels — kind of, the content level and the pure UX interactivity level.

      Augmented audio possibilities

      Sonal: Okay. So we can shift into discovery and recommendations in a bit. But before we close this thread, what do you guys think of this trend and phrase — augmented audio? Which means different things to different people. But the idea that you can actually, to your point, Connie — much like video has many layers, you can actually bring more and more layers into audio as well. Do you guys have any quick thoughts on that?

      Connie: Oh, so many. But that really just leads me to the belief that audio today is still this more “sit-back” experience. It’s very much like a one-way consumption experience, the same way that we consume television, or the same way that we consume movies. And, kind of, like — more YouTube, live streaming, that kind of format hasn’t really arrived in mainstream and audio yet. And so even just capturing the comments — the feedback to podcasts — like, that kind of content is not well harnessed today. So there’s so many more layers around the listener feedback, or interacting with other listeners, or interacting with the creator. A lot of fun should be added on and layered on into audio that, right now, at least, doesn’t exist.

      Sonal: It doesn’t have to even necessarily be fun. I mean, as a creator, I found the news — when you guys rolled out your polls feature — to be quite interesting. Because we just had the debates here in the United States, and I literally was like, “I wonder if a lot of the political news shows should do, like, their own polling as part of their audio experience?”

      Connie: I mean, it’s not just fun, it’s instant feedback.

      Gustav: Yeah. I agree. We started with PULSE which is both a safe and constructive way to bring feedback. You mentioned the consumers or the listeners talking to each other. You mentioned the creator talking to the listener. We try to focus on the creator, and what tools does the creator want? And, actually, not just for having fun — but to your point, Sonal, to be a better creator. What information do you want from your fans, and what would make it easier for a creator to produce another episode, for example? And so we started with PULSE, which is one way to get clear answers on questions you have. And we want to continue in this way — focusing, not really on listening to listen to conversations. I mean, you have Instagram, Facebook, Twitter — there’s lots of places to go and talk to other users, but there aren’t a lot of places to have good conversations with the creators.

      Connie: And I think if you focus on creators, there’s also a huge opportunity to expand the funnel of creators. If you look at trends in video, lots of the top trending YouTube videos are actually reaction videos, where people are watching a video and showcasing a reaction. And TikTok is all about remixing. There’s a lot of great audio content out there today, that if you talk about augmented audio — you could take a podcast and then have another person share their thoughts directly, just like a sports broadcaster, even — commenting directly on what’s happening in the audio, whether it’s music or even another podcast.

      Gustav: Yeah. You have these two extremes like the old-world broadcast, one-way media. And then on the other extreme, I would put gaming, where the interactivity is the experience. You’re not being broadcasted anything, you’re actually creating it. And then you have this thing in between. And I think audio needs to move towards interactivity. And like I said, there is basically a cheat sheet where you can look at other types of media. And as soon as you add a feedback loop, the creator gets a chance to improve. So I think that’s vital.

      Sonal: Tell me more about some of your thinking behind polls. When you guys design a product, do you actually have an opinionated philosophy that, “This is how we think people are going to use it?” Or are you just giving them the bare minimum and then unlocking your community to, kind of, let loose? A simplified way of asking that is also, is it a Steve Jobs point of view, or a Bezos point of view?

      Gustav: That’s a great question and a great way to put it. And it’s a tough question to answer. It’s definitely not a Steve Jobs point of view, in the sense that we know how people are going to use it. But we try to be slightly more opinionated. We don’t have the complete bottoms-up, or throw stuff at the wall. I think it’s due to our history. So, when we’ve developed products in music, it usually involved — once you came up with the idea, you had a three-year roadmap to go and license that idea from four majors. And if you licensed the wrong thing, you lost four years. So you needed to be right, and you needed to be more sure, because the cost of being wrong used to be so high for us. And I don’t know if it’s good or bad. I think if we had grown up in a world where the cost of being wrong was just the engineering time put into it or something, and you can just pull it back, maybe we would be different. But we have a pretty specific culture where we actually do plan quite a lot more. I wouldn’t say Steve Jobs, for sure. And Daniel himself actually talks all the time about distributing decisions, but it is more opinionated.

      And then for PULSE, we’re lucky enough to have Gimlet and all these studios in-house, with lots of fantastic creators. So we get to test this internally, and we use them as an internal inspiration. And sometimes they are the product owners, because they represent the user needs.

      Sonal: That’s fantastic. Connie, more thoughts on interactivity? I feel like you live in this world, and you talk so much about China apps and what’s possible when it comes to interactive audio.

      Connie: So another interesting thing about creators that comes from looking at what’s working in China, is not just giving them feedback on what the audience wants to hear next, or what the audience is thinking. But also separating your average listener from your super listener — the person who really wants to, even pay you directly for your work. And helping you identify who your real true fans are, right? If you think about the creator economy — very clear trend that’s already been in Asia for a while now.

      So, something like the QQMusic, which is the main music app that people are using in China. If you have someone who is hosting a radio show or, kind of, a listen-together type of group chat, there’s the option to, basically, be part of their paid fan club. And then if you’re a part of their paid fan club, you get a different badge on your own profile, you get access to exclusive virtual gifts that you can send that host — so everyone knows that you’re a part of that paid fan club. You can get a different announcement when you enter the room, different kinds of bonus check-in tasks. There’s a bunch of new features that get unlocked if you’re a part of this creators’ fan club. And, ultimately, what that allows the creator to do is monetize better than just a traditional advertising route. Because in addition to receiving normal virtual gifts from their listeners, from anyone who drops in and participates, you also are cultivating your small following of super fans who really, really love you.

      Sonal: I love that you’re pointing that out because it’s basically making this link, that these tools and features are not just about getting more information or data— but, actually, they’re paths to monetization as well, which is super interesting.

      Connie: Well, it helps you create your own empire in a different way. Like one feature I love is this battle feature, where you can almost battle another radio station at the same time, and almost compare how many gifts each of you are able to aggregate in a certain period of time.

      Sonal: It’s like duets with an audio challenge.

      Connie: It’s really focused on how to help creators motivate their community and build that core fan base.

      Gustav: So, one of the things that I think is really interesting with these things that you mentioned — they’re dependent on actually having a logged-in service, so that the creator can understand their audience. That wasn’t really possible over the previous protocols. You got download numbers, <Yeah.> but you couldn’t really understand your audience and who was your super fan. You know, what they look like, and who they are, and where they live, and so forth. Whereas, that protocol doesn’t actually support feedback to the creator — it’s a one-way broadcast protocol. 

      But because we’re now, sort of, full stack, we can start doing these things that have happened in other industries. And the thing that happened in video, and in many of these other things — like, you take text messaging, for example. It used to be [that] standardized and innovating on that text messaging protocol needed a ton of carriers to sit in different forums and agree, right? So the benefit was ubiquity and reach, but innovation was really slow. And then at some point, something like Snapchat happened, that verticalized the whole thing — and, you know, WhatsApp and so forth, and innovation just ran away. One day, you had disappearing messages, the next day you had stories, the third day you had lenses — because it didn’t really have to wait. And so, I’m really excited about that happening to audio.

      Connie: Yeah. This is what we mean when we say, like, very early innings of audio.

      Gustav: Exactly. But there was, like, a technical foundation that needed to exist. That does exist in China, to your point. They’re all vertical.

      Sonal: Yeah. I’ve been very obsessed with — and the student of — the history of innovation. And to me, this is the classic arc from when you go from a utility layer to, like, a value-add layer. And, of course, there’s a lot of debates around what platforms should and shouldn’t have control over. And that’s something that’s playing out a lot with crypto, and a lot of other discussions. That said, I think the point you’re making, Gustav, which makes it less academic and more interesting to users is — it is really — comes down to — you are giving me something I can’t get right now.

      Connie: Yeah. If you have one app that can give you a vertical solution — basically, give you everything you want — that app’s true understanding of you is very strong, and its ability to personalize things towards you is higher. Your ability to create a profile, that you then are proud to share with other people, or that you want to build upon — whether it’s earning different levels or different points, that also increases.

      I mean, I love what Gustav is saying about how things are more vertical. There’s a lot of benefits when you take, kind of, the super-app mentality. And a super app is basically a product or a platform that focuses on all the different needs a particular customer wants, versus giving a single-feature solution. Recognizing that, “Oh, this person loves listening to these kinds of music, but this person also probably loves listening to all these other things. So why not let’s offer this all-in-one package? We now better understand that listener, and we can solve more of their problems.”

      Gustav: So, we were actually quite inspired by the super apps of China when we thought about podcasting. The obvious solution, if you’re going to build a podcasting app — if you come from a pure design angle — is to build a standalone app. But the trade-off, then, is distribution. And so, we looked at it more from a super-app point of view. And we realized that what users actually wanted was all of their audio — you know, which they used to have on radio, music, and talk, and so forth mixed. And we had a zero-user base in podcasting, so we’d be starting from scratch. We had hundreds and millions of music users, and that’s an advantage in itself. But more importantly, we understood these users. They were logged in, and so we could just augment their moments. And one of the interesting things we found was that it turns out that your music listening is actually very predictive of your podcast listening.

      Connie: You can probably guess a person’s age range from their music listening alone, right?

      Gustav: Yes, you can. For sure.

      Sonal: So, you’re saying people’s music listening predicted their podcasts taste?

      Gustav: Yeah. When you want to cold start a podcast listener, it turns out that your music listening is actually a really good signal for that — for which podcast you recommend.

      Sonal: That is incredible to me. I just think people’s music listening is so much more visceral and less intellectual — that I’m just so shocked by that fact.

      Gustav: I would not say it was obvious to me either, but it’s, like, a very clear result. It also supports the idea of the audience — that you should think of them as one person, right? And try to serve them in the different needs they have.

      Connie: Yes, think of the customer as one person.

      Challenges of integrating media

      Sonal: Right. What you’re basically both really saying is — when you think of the super-app mindset, it’s a cohesive identity of a user’s needs. And, in fact, if I were to visualize it, I think of that classic Da Vinci Renaissance man [Vitruvian Man], where you have like this person at the center, and then you have multiple spokes of interests — kind of, radiating around them. And then you think of each of these moments in their day. It could be time, it could be interests, it could be need. It could be whatever job to be done, to use a Clayton Christensen framework — and that you’ve referenced a few times, Gustav. But what you’re both also essentially saying, is that a super-app — once you have one — is built in distribution. And so you’d be silly not to use that base and do the cold start.

      Gustav: Yeah. It’s much easier to say, “Let’s put a competing team over there and let evolution take care of [it].” They build their own app and they compete. But it’s at the cost of the user to do it that way. And so the first thing we did was, we figured out that instead of having the apps be as different as possible, you actually wanted to have them be the same thing. And you can say that radio has always done this. People have been mixing these mediums, so it didn’t seem that far fetched. But it wasn’t clear. And if you optimize for ease of implementation, you have small things such as — just the fact that the UI has to change from skipping a whole song, when you’re listening to music — to, all of a sudden, skipping 15 seconds back and forth, and scrubbing within a podcast. That’s a big challenge to solve dynamically in the same UI. It would have been much easier to just maximize the two different hypotheses.

      Sonal: Yeah. So, basically, what I’m hearing is, even something as seemingly mundane to the user as the ability to scrub forward 15, 10 seconds — which I do all the time in my podcasts. If you’re in music, you can just skip an entire song forward. And even that kind of trade-off is, like, actually really complex when you’re doing it in the same UI. That’s super fascinating.

      Gustav: Exactly. So the UI has to be much more dynamic.

      Connie: I mean, even how you show a track versus an album cover, right? Or a podcast episode versus the podcast cover — like, it’s a very different thing. It’s not easy to pull off. And it gets harder and harder the bigger the company is, because it requires real changes that are top-down, that have to come from leadership. It’s a change in your org structure, it’s a change in your release cycle. It’s a massive change, and it’s very hard to pull off.

      Gustav: It was painful. We needed to “force.” It’s not like people didn’t want to do it, but you needed to get people to work with each other instead of putting [it on] a different team. And it certainly needed global prioritization, from Daniel down. And we have this system to prioritize things globally, called <inaudible> in Spotify, which was very helpful to get these things through the company. And I don’t think if we’d had that global prioritization tool, we could get this through the company. It’s very hard to do. But this is the benefit of software, right? And this is one of the benefits of being full stack. We can actually try to solve these problems, and actually improve the consumer experience.

      Redefining audio as media types merge

      Sonal: So, let me ask you guys a quick question — especially you, given Spotify worked within the existing UI to blend from music to podcasting. Where do you stand on the definition of podcast, music, audio? I always talk about how audio is a huge category. Like, I honestly think trying to homogenize audio is like trying to homogenize text. It’s like — a word is the same thing as a book, is the same thing as an article, as a blog post, as a tweet. That’s ridiculous. However, Connie, you made the argument in our podcast about podcasting, with Nick Quah — how podcasting and music— and I agreed with you, as well, then — that there’s a big difference between the spoken word and the sung word. And so I’d love to hear your guys’ thoughts on, where are we today?

      Connie: Radio is the integration of both talk and music. They live very symbiotically together. And if you look at most podcasts, they have a music introduction already. There are sound effects in a bunch of them too. So this combination, or this belief that normal talking can be improved with music, or music can be improved with talking breaks, has been here forever.

      Sonal: But even then, where does, and doesn’t the blending of music and podcasting actually work, and where does it fall apart?

      Gustav: Right. So we had this intuition that people wanted their music and their podcasts in the same app. And that certainly turned out to work. But there was a category where they’re actually related. It is the same session, right? So this is the thing that we just released. So now we are going to let creators do this new type of session, where they can mix talk with licensed music in a seamless session.

      So, you see these two user needs. If you take the Clayton Christensen approach, you see podcasters really wanting to use and talk about music, but they can’t — because the creators do not get paid for some burnt-in song in a podcast. And then you see the music creators that would like to talk about the music. So you have both of these sides at the same time. And it’s been really hard to solve it, especially if they were two different apps. But now it feels very natural that you should be able to have this new type of show.

      So you’ve seen us play around with things like Daily Driver, for example, for a long time, where we mix talk and music. And we’ve seen a lot of success. People love hearing their news, and then their new music in the same session. Especially when they’re driving — trying to switch to the music session and hear the new releases as well. But so what we were thinking now is, we want to enable anyone to do that.

      And on the consumer side, it is neither a podcast nor a playlist. It’s just, <Yep.> the best of podcast and the best of playlisting. But it is neither, because podcasting has the problem that you actually aren’t allowed to feature music in it — and playlisting has the problem that you actually can’t comment between the tracks. So we created this new format where you can do some talk, then you can add a Spotify track in there — then you can do some more talking. And so the user can then listen to the talk part as if it was a podcast. They can listen to the track, they can skip the track — but they can also save the track if they like it. One of the things that radio has missed. So it’s a new format. But, hopefully, it’s not new in the bad sense, that you have to learn anything new — it should be just like listening. Because, then it works the way you, kind of, always wanted it to work.

      Sonal: What would you call this new format? I think very broadly of, again — I mentioned how audio is as heterogeneous as text, so it’s ridiculous to use one word for everything. But it is a new kind of audio experience. It’s not a podcast, it’s not music, or a song.

      Connie: I think of this as going back to radio. For me, this is the new radio station.

      Sonal: Yeah.

      Connie: This is the new way you can listen together.

      Gustav: In a sense, a very obvious innovation — but also an innovation that requires tons and tons of licensing work over many years, and a big investment in podcasting and creator tools and so forth.

      Connie: I’m smiling because it’s going to open the door for a whole batch of brand-new creators. People who don’t want to host a podcast and talk the whole way through, but now can use music as their passion — as their content — as the thing they’re, kind of, anchoring their talk around. And then this also brings about curation, social discovery. I mean, I can even think of several a16z colleagues, myself, that I think would be really good creators on this new platform.

      Gustav: That’s what I’m hoping for. I’m hoping for you, Connie.

      Sonal: I think she means Anish, because Anish is a side deejay.

      Connie: No, my stuff will all be probably Chinese music.

      Gustav: We want that too.

      Connie: Yeah. But the point is, it really opens the door to new batches of creators. And it brings in social discovery, and it brings in the idea of curation. It’s back to, kind of, the Spotify playlist, but with more color, right? And with more storytelling.

      Sonal: Augmenting, I might even argue.

      Connie: And the interaction that you can have with the listener, right? In Asia, you can have people order different songs and pay to try and see what’s already on the playlist, and change that playlist — even in real time. So the kind of interaction you can build on top of this is also exciting.

      Gustav: And you spoke about augmenting there, and I think that’s a great point. So we spoke about TikTok, and I mentioned this pattern of taking a, sort of, commodity licensed music and letting your users make it unique. So one way to think about this is, it’s a similar pattern. We’ve had tremendous success by letting our users work with the music catalog and playlist it. You know, they create billions and billions of playlists that have helped them, and has helped other users. But it has also helped all our algorithms to learn, right? So you can think of this as a similar pattern, where you take the commodity catalog, but you let any creator, through Anchor, work with it and make it more unique and uniquify it, right?

      Sonal: I love it, uniquify again. Well, the other interesting point is when Eugene and I talked about TikTok on this podcast, he did bring up that one of the big unlocks, as minor as it might seem for the remix culture as well — was the ability to quickly license, combined with the creator tools, combined with the distribution — so that you do, then, get this “creativity network effects” flywheel. Which, sort of, then reinforces.

      Connie: Yeah. It’s a big way that people are interacting with music on the QQMusic app. When you tap into radio stations or listen together, you see all these different hosts, and you can listen to them live. When you’re listening together with other people, you can choose different topics or categories — like friendship, music, emotions, talk shows. And the interactions that you already see happening on these radio stations are “listen together” — there’s a chat that’s usually going on while people are listening to music. There are different leaderboards for these different creators. You can have different tasks that the creator asks you to do. You can order songs, you can see what’s next on the playlist. You can gift the creator, and thank them for curating this kind of music. And you can even subscribe to their fan club, right? Like, if they always have great music choices, you can make sure that you’re always able to know when they release something new, or when they go on. So it does unlock a brand new batch of creators that today don’t live on YouTube. Today they’re not podcasters. But they have a lot of things to say, and they love music. So a lot more people will be able to participate — be creators themselves, build a following, and eventually monetize.

      Gustav: I agree. The increased participation of new types of creators is really interesting, because there are all of these creators who clearly want to talk about music, and there are all of these artists who, you know — they’ve always wanted to be on radio. <Mmhmm.> Like, they want to be featured by someone, but business models [are] often a problem. No one has been able to solve that, [so] both parties actually get paid for that. We solved what I think is a harder part, actually — of licensing all the music in the world and paying royalties to all organizations. We’ve already solved that, so it feels like a very natural product for us to play with.

      Connie: Yeah. When I was growing up, I used to listen to radio shows. You know, I used to listen to Delilah, and she would have stories in between. And then she would have audience people call in. And then she’d have nice, soft music to go with that story.

      Gustav: Exactly.

      Connie: And it was fantastic.

      Gustav: And then you probably recorded the tracks, right? Because you really wanted the music?

      Connie: And that’s how I discovered music too, right? And that’s how she could also resurface music from the past, rather than having us listen to only stuff that was released in the last 18 months. Let’s resurface some of these oldies, and this is potentially a great way to do that.

      Sonal: What’s really fascinating to me about this is, it’s almost like a vector to social. Because there’s nothing more inherently social than music listening, and music sharing. As you’re noting of playlists, music curating — and to your earlier points about it — unlocking creators. One of my favorite podcasts, actually, is “Song Exploder” by Hrishikesh Hirway. And I actually think I heard about this podcast from Eugene, actually, like a year ago. And it’s now they’re going to be a Netflix show. And, you know, he really deconstructs these songs on air. But imagine all the people — like all the kids, all the adults, who just lie around listening to music, talking music with their friends, bonding over music. So, to me, what’s really fascinating here is — there is a social vector, both socially and para-socially with acquaintances and strangers, when you think about them connecting with fellow fans of those playlists and other people. So I think there’s actually a really interesting vector to all that too.

      Gustav: Yeah.

      Sonal: Because TikTok is not a social network, but this theoretically could be.

      Gustav: So this is an interesting point. We think about Spotify more like YouTube and TikTok, than Facebook and Twitter. It’s actually not about following your friends, but I think you’re right. I think there are so many creators out there who would love to tell a story about a specific piece of music, right? Their own story, some story, or something. And we’ll see how it gets used. I’m hoping, obviously, that many artists would like to tell their story of their own album that they released, for example.

      Sonal: Yeah.

      Connie: Yeah. Amazing.

      Gustav: There are many different things that could happen.

      Connie: Even in that great example where the artist is telling the story, that artist doesn’t have to sign up and say, “Okay, I’m going to start a brand new podcast.” That is such a big responsibility and commitment to take on.

      Gustav: Exactly.

      Connie: And now you, kind of, have these, kind of…

      Sonal: A Trojan horse is starting a podcast <inaudible> basically.

      Connie: This really lowers the bar of commitment for creating a show. And you can try it with no real consequence, and get that distribution, too.

      Recommendation algorithms

      Sonal: Okay. So, now let’s, then, talk about — how do you solve — this is, like, the big elephant in the room — and, potentially, the big exciting thing in the room — recommendation and discovery. How do you, then, think about that side of this? Both in the context of Spotify shows, and also beyond. We opened this conversation about what has and hasn’t changed. This has been a broken problem “in podcasting.” It might not be as broken in music. We’ve talked about TikTok, we’ve talked about the parallels and differences between video. Let’s bring it all back together around this theme and topic of recommendation and discovery.

      Connie: For music, there is a commitment of more than two or three seconds to figure out if you like a song, right? So, the bar for who you trust as your source for who is giving you that recommendation is higher. And so you either have to have a system that builds trust, showing that their algorithm has given you enough hits. Like, TikTok can’t be wrong five times in a row. Stakes are really high. So you either have an algorithm that is so good that it knows enough about you already — that the majority of the time, when they give you something, you like it. Or you have a creator that also has that same kind of hit rate. That you realize, “Hey, most of the stuff that that person likes, I also like.” And that is also a great way to, kind of, get that discovery element. It’s all about giving the user this end trust — that they’re willing to test your recommendation because, say, 80%, 90% of the time, you’re going to be right.

      Gustav: So I think you’re completely right. That was a success with user playlists. There are literally many billions of different curations of the Spotify catalog, so you literally have something for everyone. And either they find that playlist, or you can use machine learning to learn from that to be able to serve users. Then you have the UI elements themselves. And I think that’s different between music and podcasts. Music is easier, in a sense, because it is three-minute items and you can skip through. And what we see in music is that it’s, like — the investment of how much time do you spend, versus finding one jam. So it is actually okay if even most of the songs, theoretically, are not that good. If they’re easy to skip through, and, like, the seventh song is, like, your dream song. Because that can make your entire week, or maybe month, right?

      So I try to think about it — I think Chris Dixon said this, “a fault tolerant UI.” If your machine learning is perfect, you only need to unshow one item. If your machine learning is 1 out of 10, you probably need to show 10 items, because then there’s always one jam on the screen. You have to adapt your user interface to your, kind, of level of recommendation. And so, these playlist formats — we try to think of it as, kind of, a GTD — get things done. Can you quickly go through and like, “Yeah, that was perfect, save that to my library.” It’s like a productivity flow in the discovery moment, which is very different from the consumption moment, when you may be on a speaker. And then it’s not okay that you have three bad songs in a row, but it’s okay if the fourth one is good. Does that make sense?

      Sonal: That goes back to modes, actually. Thinking about the mode the user is in.

      Connie: Yeah. I also think, if there are good mechanisms in there for the creators to have potential financial payoff from participating, the creators are actually going to be incented to have discovery. That incentive is actually built in. Because you cannot have thousands of concurrent Spotify shows all showcasing the same music. No one is going to want to listen to that. And so, all these creators are naturally going to be incented to showcase you something brand new, because what they’re really being valued for is their ability to curate, and then match that with the storytelling.

      Let me give you a concrete example. When I go to the gym and someone is trying to do a workout, and they’re talking through, and they have music sliced in between. Or just think about a yoga class — they want that variety of music. They don’t want you to be listening to the same thing, time and time again. And now even that gym workout, that yoga class — could exist as a Spotify show, where they’re making you do pushups and counting down, and then there’s music right there in the background. You have to really think [about] what this can unlock.

      Gustav: I’m definitely hoping for that yoga and pushup workout to happen. You have to make it happen. <laughter>

      Sonal: Okay, Connie. So either you make a yoga show, or you do, like, a Chinese song playlist.

      Connie: No. But the point is like, there’s so much context that can now be wrapped around recommendations. Like, even the time of day — what are the right kinds of shows that work for the morning, what are the right kinds of shows that you want to wind down to. Those creators will have the incentive to naturally pick what they think makes sense for you.

      Gustav: Exactly. So I think there are two things that are really interesting here. So one is, when we think about machine learning overall, and recommendations from a product point of view — and this is completely borrowed from Andrew Ang, by the way, so it’s nothing that we came up with. That we try to use is — if you think about what algorithms do really well, they tend to scale really well. They tend to be able to personalize, at an okay level, to hundreds of millions of people. Humans don’t do that really well. Humans are incredibly smart and creative, though, but they don’t scale so well. So one way to think about this, that I think Andrew Ang coined — was to let the editor, for example, or the creator, if we’re talking a Spotify show, but an editorial playlist — this algotorial principle that we use.

      Sonal: Algorithm plus editorial.

      Gustav: Exactly, algorithm plus editorial, that we call algotorial. You literally think of the editor as the product owner. This is the product person that has the idea and the hypothesis. And they come up with what the job to be done is, or what the hypothesis is, or what the use case is. So, for example, you take something like songs to sing in the car. No machine came up with that idea. It was a human who sat and said, like, “I think there’s a user need here. People want to scream their lungs out when they’re driving to work.” So how do you teach a machine this? The algorithm doesn’t understand what “songs to sing in the car” means. Is that, like, a bit of ’80s music, is it a bit of movie music? But for a human, it’s super clear — like, this is a song to sing in the car, this is not. So what the editor does is, they literally create, like, a playlist of a few thousand tracks, and then the algorithm can understand it. And they can personalize it to 300 million people and scale it, right? So the job of a product owner is to create this data example, this data wireframe — I think, is very useful. That loop has been very useful for us.

      Sonal: So, basically, bundling the best of human creativity with the best of algorithmic scaling, in order to deliver on the personalization and recommendations to a mass of users.

      Gustav: Exactly. Humans have to come up with the ideas. They have to show the ML system what that idea actually looks like for the ML system to understand it. Because the ML systems are great at scaling, but not great at coming up with new ideas.

      Sonal: Can you give me a little bit more color on some of the challenges here? I’d love to hear about how you have to think about solving them — what’s hard about algotorial. But then more specifically, about how you had to negotiate that, when you transitioned from music to podcasting — and then now in blending the two. I want to hear a little bit more color about it, basically.

      Gustav: So, in music, we have, really, two sources — traditionally — of recommendation information. One big source is the playlists, the other is editors. But then we have the third way, obviously, which is the engagement from the users — listens, and skips, and so forth. Those are the signals in music. But music is different because the items are three minutes long — like we spoke about, it’s more like TikTok. Then you go to podcasts, and it’s like maybe one and a half hours, and then you get one skip. <laughter> It doesn’t fit at all with, like, “Let’s just, you know, feed the machine,” right? It’s very low signal. So we had to think about it completely differently. But, not only is it much further between the skips — we don’t have anything equivalent to a billion playlists. So we had to go back and start working with “old tech,” like knowledge graphs.

      You have other advantages in podcasts which is — there’s actually information in the audio. You have other signals. You have show notes, and you have the transcripts on the shows. So we started working with those technologies instead to get some understanding. So, actually, these two stacks are quite different. We certainly could leverage a lot of learnings, but they’re not the same thing because there’s such different objects.

      Connie: Especially because podcasts are usually multiple people on a podcast. There’s oftentimes a host and a guest.

      Sonal: You actually don’t know who people are following sometimes, who they care for.

      Connie: You don’t know. If there’s, like, a Joe Rogan talking to Elon Musk, you don’t know if it’s because I like Elon Musk or if I like Joe Rogan. That’s quite different than music, where there’s a bunch of artists — any song they put out, I’m going to like, I’ll take a listen to.

      Sonal: It’s like a cult of personality show, because you’re following the host, in that case. In this case, you’re following the artist. But one thing that I think is really interesting when talking about the knowledge graph is the mood graph. I always talk about — coined the phrase when I assigned an op-ed on it a number of years ago at WIRED. Because I actually think we’re missing a huge opportunity in optimizing things. Frankly, my playlists are all organized by mood and emotion, they’re not organized by any other criteria.

      Gustav: That’s a great point. And in music, that is one of our biggest vectors. Like, one of the biggest sections of editorial playlists are the mood playlists. You’re completely right.

      Sonal: Ooh, that’s great. It’s interesting you bring up a knowledge graph, Gustav. Because it’s tough to know — is it a book author? They’re just listening to every single podcast they’re on? Is it a content thing? It’s so complex and multi-dimensional.

      Gustav: Exactly. And the answer, as far as we can see is — it’s all of the above. There’s “personality cult,” there is, you following a certain guest around all the podcasts that they visit. There’s interest. It’s just going to computing — I don’t care who is talking, right? So you really need this knowledge graph with all of those dimensions, and then you need to be able to let the user, kind of, traverse along these different dimensions. And then you can lead them to some discovery. You remember this debate around music — everyone had a music friend that influenced them. And for a while, early Spotify, we invested heavily in social to try to replicate that. But it turned out that most of your friends on Facebook — they don’t inspire you so much musically. If you average them, it’s just the U.S. billboard. So we take the same approach in podcasting.

      I mean, we have a core belief that if Spotify can make you discover something that you wouldn’t otherwise have discovered, it will be more important in your life. So we really try to make sure that we measure and understand how many discoveries we generate for you.

      Sonal: It’s almost like a new metric of return on discovery. Instead of return on investment or return on energy, if I think about every app, what is my return on discovery — or ROD — on that particular platform?

      Gustav: I’ll borrow that from you. But another difference from these things is that we are, revenue-wise, mostly a subscription service. So in machine learning, in the practical world, there’s been a lot of deep learning and so forth. But in the academic world, for a long time, there’s been a lot of focus and discovery and exciting results around reinforcement learning. But, you know, AlphaGo and all these things.

      Sonal: Yeah. We’ve actually talked about it on this podcast quite a bit, too.

      Gustav: And not to go through it. But the main idea is just — you look for some long-term reward and you backpropagate it through time — instead of looking at, what is the most likely next click? And so, I think, if you have a service that is free only, and, you know, you have an average engagement — same every day — it’s going to be really hard to, like, backpropagate signal. It’s going to be noisy. But if you have an event four months down the line — that is, you know, I went from just consuming ads to paying $120 per year, you have this massive amount of, sort of, gradient you can backpropagate through time.

      Sonal: Oh, I love this.

      Gustav: And the thing that is different between, for example, YouTube, or TikTok is — every month, all the paying users, hundreds of millions of them, they go and they evaluate. It’s like, “Should I still pay?” And they vote with their wallet regardless of how much they actually consumed. So we have a different signal that is not just engagement, and consumption, and attention. We can see — do you keep paying? And, obviously, as you know, it’s not really possible to do the real reinforcement learning. You basically need a perfect simulator of the world. But you can approximate it quite well. And so that’s something that is happening in the rest of the industry as well, slowly. You need enough signal for that to really be valuable. So that’s something I’m excited about in the recommendation space.

      Sonal: What you’re basically saying — I talk about this quite often on the podcast, about how subscription models change so much. But what you’re saying, which is so fascinating to me, is that it’s also a way to get much better signal into your system.

      Gustav: Right.

      Sonal: You’re also basically saying — you’re essentially weighting higher people with more skin in the game, which is exactly how you want to design something.

      Gustav: Exactly. Everyone has saves and likes. But you can think of, like, paying $10 as a super big like every month.

      Sonal: Yes, exactly. You’re weighting it higher. And you have that data because people are logged in and they’re streaming. One of my favorite books is James Carse’s “Finite and Infinite Games.” And he just died, actually.

      Gustav: Yes.

      Sonal: Rest in peace, James Carse. But the idea — what you’re saying is, you’re playing a repeated game with your users. Which then gives them an even better game board to play on, versus a transactional game only.

      Gustav: That’s exactly it. Which is a big problem that is important to solve, I think. You can try to understand what the user actually values long-term versus just in the moment.

      Connie: Yeah, subscription fees is a fantastic business model. But also, I can see how that would allow new revenue streams for these creators. And I’m not just talking about the people who create the music, but I’m talking also about the people who are going to create and deliver a brand new experience that lives on top of the music. If those people can find some kind of financial payoff in participating, that’s a brand new revenue stream. And then think about the possibilities — the kind of interaction you have with that listener at that moment — is another area you can charge for.

      Sonal: I also love that. While we’ve talked so much about putting the power back for creators, it really does actually most empower the listener. Just one quick question, Gustav. How do you think about the tension between data, and all the data you’re getting, and all the signals, and where it goes too far? Like, is there a risk that sometimes, listening to your users, you’re missing out on what they don’t tell you? And how do you think about that as a head of R&D at a company where you’re not just abstract R&D — you’re actually building product?

      Gustav: Yeah. I think that’s a fantastic question and really hard to answer. It is an age-old problem. I think one way to think about it is to simplify it a little bit. Algorithms — they, kind of, look in the rear-view mirror and draw a straight line into the future. And so, that’s great for a while. But product development — usually, good product development is based on some sort of ideally contrarian hypothesis. And your machine learning is not going to come up with a contrarian hypothesis, right? So you need some mechanism for that to happen. And so, we try to think of this in different ways. I mentioned algotorial, where the editor actually has the ability to say, like, “No, I believe in something different.” So we try to build in this mechanism where humans can go in and, you know — they have the steering wheel, they can take a left turn or something, and then the algorithms follow.

      And, you know, there are incentives to not do it. It is always going to be safer to keep going straight for a while more. Why take risks? All of these things, right? But back to playing infinite games. If you play the game, you know, many times — think about it as game theory — now you have to end up in a place where the optimal thing is to try new things every now and then, to try to cover as much space as possible. And as I said, we have a culture of being quite specific in the hypothesis we have. And we try to think about it, as do many companies, sort of, a portfolio. I want to have some things that are quite contrarian, and [have] a pretty high chance of failing. Whereas, I want a bunch of things that are obvious. But that balance — I mean, no one has the perfect solution, but everyone at some scale has to start thinking about it.

      And so we found a few mechanisms that were useful for product development. One was to take the concept of simple prioritization, and the Kanban board, all the way to the C-suite. You know, everyone thinks they’re good at prioritizing, but they’re not. And I bet that in most companies, the C-suite is the worst at prioritizing. They actually want to do everything. And so, we have something like five to seven things that the company needs to do. And Daniel owns that. But the one rule is — two things cannot have the same priority.

      Sonal: It reminds me of the Steve Jobs bio anecdote, where at one of their off-sites, they put a whole list of things, and he literally crossed everything off the list and they only did the first four. What you’re describing, though, is not just siphoning off what to do versus not to do, but what to order the priority, from the top, so that the managers don’t have this friction and they don’t waste in terms of building things.

      Gustav: And that’s the trick.

      Sonal: Yes, I agree. And the other thing that I think is fascinating about that, is that when you say that Daniel, kind of, owns that too — when you are disrupting yourself, so to speak — like, when you went from music to podcasting. Putting that higher up on the bets board in his office is, like, “Hey, no complaints, guys. This is it.”

      Gustav: So that’s exactly what happened. Podcasts was the number one company bet for two years, and everyone in the company knew it. And so, what happens if you don’t have that? You push that decision to managers and you create conflict in your world. The truth is, Daniel can’t have any idea, in a company of thousands of people, what is going to clash with what resources.

      Sonal: Of course.

      Gustav: The only thing he can do is, like, when you clash, this is the priority.

      Sonal: I love that as a management thing.

      Gustav: It’s so simple. Everyone thinks it’s so complicated. It’s actually very simple. It’s — the discussion is hard. Actually, prioritizing is very hard.

      The future of audio

      Sonal: Okay. So we started with talking about where podcasting has been. We’ve gone through what’s shifted — the parallels and differences between video and music. We’ve talked about the trend of interactivity, and augmenting audio in different ways. We’ve talked about recommendations and hearing like an algorithm, even, and an editor. What do you guys think is, sort of, the future of a lot of these? Like, where do you think the future is, kind of, going?

      Gustav: My guess is that if we use the cheat sheet of other media, I think audio is going to increase on the creator side just like the other mediums. I think it’s going to increase in numbers of creators.

      Sonal: The market for audio is bigger than I think people realize. Or, as Connie said earlier too, we’re still in the very early innings. So my obsession is this two-word phrase that I use all the time — of world-building. And to me, one of the missed opportunities in audio for a long time — and, you know, Gustav, you painted this range from gaming models, all the way to music models, to different things. I actually think we’re starting to increasingly see more game-like behavior in audio. And I’m so excited for that kind of world building. 

      But it’s a very different kind of world building, because audio has an immersiveness that’s very different than the visual-based world-building of other worlds. And so I’m super excited for what we can do. I mean, I already think about our expanding podcast network as a form of world-building. And when you’ve mentioned Spotify shows, that, to me, is another form of world-building, because you’re essentially bridging different worlds and creating new experiences. And so, to me, that’s actually the thing that I’m most excited about.

      Gustav: So I think that’s a great way to think about it. And you think of the music world, the podcast world — and now you can think of this new world where you can mix them, and then you can have other worlds. The thing that I think is going to happen is, you look at something like audio — and it’s so easy to create, it’s even easier to create than video. So, as we both make it even easier and lower the friction for everyone, we let creators make more money and we add these new formats. What I’m hoping is that that market is going to grow as well, just like we’ve seen the market for creators growing in other media.

      Connie: I think audio will be further optimized in the sense that you can almost peel apart the different nuggets of a podcast, right? You can take certain segments now. You can take a commentary around it now. And you’re going to be able to do new things when you break apart a song, when you break apart a podcast, and you can see what that will unlock. TikTok is breaking apart a song — kind of, getting to a specific 5, 10 seconds slice of it, right? A snippet. And then, this idea of now taking something that used to be, you know, one piece of content and chunking it down to different things — now [that] gives you new building blocks to build new kinds of shows, new kinds of interactions. Which means things will get much more participatory. More people can become creators. More people can probably become listeners. More listeners will find each other, listeners will become stronger fans of their creators. So I think there’s a very hopeful, very optimistic future, where now technology actually can help everyone win.

      Sonal: That’s fantastic. I love that. Gustav, Connie — thank you so much, you guys. Thank you for joining the “a16z Podcast.”

      Gustav: This was super fun.

      Sonal: Super fun.

      Connie: Thank you.

      Sonal: I wish we could all talk for hours. Take care, everyone. Bye.

      Connie: I should put in a plug for my Spotify show. <laughter>

      Sonal: The China Song Show, Connie?

      Gustav: It’s going to be huge.

      Sonal: Bye, guys. Have a really good day or evening for you. Take care, everybody.

      Gustav: Bye. You, too.

      Sonal: Thank you.

      • Gustav Söderström

      • Connie Chan is a general partner at a16z where she invests in consumer tech. She's well-known for her deep knowledge of the Chinese consumer tech landscape and spotting those trends moving from east to west.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      AI, WebRTC, Crypto, and Full Stack Startups

      Elad Gil, Sep Kamvar, and Chris Dixon

      Today’s episode is a conversation about four big trends in the tech world. Any one of these trends would be notable on its own, but we cover all four in this hallway-style chat, as a16z General Partner Chris Dixon talks with Sep Kamvar (professor of Media Arts and Sciences at MIT and now cofounder of cryptocurrency platform Celo); and Elad Gil (investor and the cofounder of health technology company Color Genomics, and formerly at Twitter and Google).

      This is a wide-ranging survey of some of the major shifts in technology right now, but it’s really a meta-story of how innovation happens, which is most definitely not in a straight line.

      So here are the trends they cover:

      *crypto (of course);
      *AI and machine learning (including GPT-3 – you can also listen to our explainer episode on what’s hype/what’s real there on our show 16 Minutes);
      *full stack startups (which Chris first wrote about in 2014);
      *and collaborative web/collaborative enterprise/ social (including RTC or real-time communication within the browser), which is where the conversation begins.

      Show Notes

      • WebRTC and WebGL and the move toward collaboration [1:23]
      • The rise of crypto [5:30] and how new technologies improve over time [8:54]
      • The current state of AI, with a special focus on GPT-3 [13:57] and how more advanced AI technology may be able to replicate itself [17:25]
      • Discussion of full-stack startups [21:33] and why there aren’t more of them in the market [25:28]

      Transcript

      The next big platform shift

      Chris: Elad, you and I have been talking about this. And I know you’re very excited about it, this kind of this feeling that there’s a new stack of web infrastructure, things like video and audio, collaborative video and audio, rather, we sort of have the infrastructure now that it works in a way that it hadn’t in the past. And that’s unlocking a whole new wave of interesting applications.

      Elad: People are always looking for the next platform and what the next big platform shift is. And I think it kind of may have snuck up on all of us in the form of WebRTC and WebGL and then related API companies providing sound or other things that then built on top by many other companies. And I think this shift is substantiating itself in two different ways and I almost call it the collaborative web and then separately, the collaborative enterprise.

      And if you look back 10 years, people kept talking about during the first social wave, everybody kept talking about how there’s going to be a social enterprise and how every SaaS product was going to be more social and collaborative and that largely failed. And it feels like that shift is finally happening in part due to things like WebGL. You see Figma, for example, is the first really strong example of a WebGL-enabled application, allowing you to collaborate in real-time with other people.

      In parallel, WebRTC is really allowing for really interesting concurrent sessions around video. And so, you’re starting to see that in terms of a lot of products being built around virtual office rooms, virtual conference rooms. And I really do think this is the moment where collaboration is finally being built into the enterprise world and enterprise products. And then in parallel, WebGL and WebRTC really seem to be enabling really interesting social experiments right now in terms of new social products.

      You have really amazing video and audio quality. So the time lag is gone so you can do things like Clubhouse. We see lots of interesting video experimentation. So you can see almost like degraded forms of VR or other things happening in browser. So, I just think now is a really exciting time of innovation around this new web stack.

      Chris: And to your point about sneaking up on us, we’ve obviously had, you know, the ability to have conference calls, group audio for, you know, decades, right? Like, the fact that they’re (now) so low latency and you’ve got like the visual representation of the room means. To me, it’s like, if you remember the old days in the conference calls how you always have people talking over each other partly because of, whatever, 300-millisecond delay?

      It’s remarkable how the conversation switches from person to person. It’s the latency. I mean, we’ve all now experienced this with Zoom, right? Like the fact that it doesn’t stutter, the fact that, you know…or very rarely does, like, it’s somehow kind of crossed over this point of good enough.

      Elad: We’re finally hitting the point now where in terms of video quality and the ability to stream concurrently across multiple users and in terms of audio quality, we’re hitting that point where the web infrastructure is really supporting the ability to have extremely low latency.

      Chris: When you call it a new platform…we’ve cited a few examples. But when you say platform, that means you think there will be thousands of examples or do you think it’s gonna be a whole new wave that goes 5 to 10 years?

      Elad: I think like any “platform,” there are going to be a handful of things that really matter that will really be the important things on it. And then a lot of things will be experiments that fail or don’t work. And I don’t know 10 years from now what’s going to be the main set of applications. I just think it is a shift that enables a bunch of new applications to be built, particularly either social or collaborative enterprise.

      One example that I think is worth noting in terms of what’s coming due to WebRTC is it’s quite possible that if you look at virtual reality or VR, the predominant use case in the near term may actually shift to the browser. And so, I think right now, in order to experience VR, you need a headset. You need, in some cases, client software, etc.

      And so, there’s more obstacles and hurdles to be able to just participate. And I think one of the things I found really interesting about WebRTC and WebGL is the ability to suddenly create VR-like experiences where you just drop any URL and you can show up. And so, the big question in my mind is Oculus almost like the desktop computer versus mobile devices, where the desktop really helps you do powerful tasks but you can do a lot on your phone and it’s sort of the mainstream use case for most of the internet today. So, I think that’s another thing that we’ll see if it happens or doesn’t happen over the next decade. But that may be one interesting long-term trend to watch relative to WebRTC and WebGL.

      Crypto, and patterns of innovation adoption

      Chris: So, let’s talk about the next trend, crypto. We’re all involved in this. Elad, you invest in crypto. Sep, you’ve co-founded a company, Celo, in crypto. Obviously, I spend most of my time investing in crypto. So, can you tell us a little about why you’re excited about it and the stuff you’re working on in Celo?

      Sep: I’ll start off with kind of a general principle that I think is true for all of the technologies that we’re talking about. There are certain class of technologies that increase the expressive range of a certain medium. And when you increase the expressive range of a medium, a lot of things pop up that were not possible before because you now are playing in a new design space.

      The historical example that I always love to point to is in the 1800s, the invention of the metal ferrule in painting — the little piece between the paintbrush and the paintbrush handle — and the collapsible easel. Those two things together allowed people to A, bring their paintings outside, and B, start to paint with a new brushstroke that allowed them to quickly dab paint onto the canvas. And those two ended up kind of giving rise to a form of painting that we now know as Impressionism.

      And so, it’s interesting to think about that. Impressionism was a result of technological advances in painting. And you see that same thing with the web and the internet in general. There were technological advances in the medium of text. And so, all of a sudden, people could send text more quickly. Anybody could be a broadcaster. You could start putting text together with code to create different things. And that vastly increased the expressive range of text in a way that led to all of these things that you could not predict in advance.

      So, for example, in ’94 and ’95 when the web was starting to become popular, one could not imagine that, “Oh, well, one day I’ll be able to press a button and order my groceries on this and have my groceries come to me,” you know? And so, I think those are really interesting from a broad-brush technological point of view.

      Why I’m excited about crypto is that crypto does this for money. It increases the expressive range of the technology that we know as money. And that I think will follow very similar to the internet. You know, at the beginning of the internet, you saw it allowed people to pass messages more quickly to one another across a distance in a way that was just qualitatively different than fax.

      And that is like the first thing that you started seeing with crypto and it has direct implications to things like remittances or banking the unbanked. But then on top of that, the second implication of the web was that anybody could become a broadcaster. I mean, with YouTube, anybody could have their own TV station. And in the context of crypto, you have the same democratization but in financial services. And so you see this kind of rise in decentralized finance or open finance.

      And then third, most exciting, is it allows money to become programmable in the same way that the internet allowed text to become programmable. And that, I think…I mean, we’re seeing some early things today. But that’s, I think, the aspect that we’re still the earliest and it has the most legs. And it is the most powerful and the most difficult to predict at this stage since we’re in such an early phase.

      Chris: My framework for this is when there’s a really big breakthrough technology, there’s two stages. And the first stage, you do things you already did but do them better. And the second stage, you do new things you never could do before. And this goes back to the collaborative web stuff we were talking about before. Like, in the first stage, you know, we’re gonna do better video conferencing, right, better audio conferencing and that will probably be a wave that lasts a few years.

      And then at some point, people will start to figure out this is a whole new set of things we’ve never done before. Like, the analogy on the web, right, is the first era in the ’90s, people were just kind of putting websites up. They were basically one way. They were brochures and magazines. But then it took another decade to realize there’s things you can just do that you could never do before like social networking, right? It’s multi-way medium, not a one-way medium, right?

      It’s similar to my understanding of the history of film. When film started off, you know, they filmed plays, right? And then they realized you could do all these new kind of film-native things, right? And I think crypto will be the same thing. And you hear…the mistake people make is they say, “Oh, great, you can lower payment fees. You can send cross-border payments.” And all of that is true but that’s only phase one, right? Phase two is things we can’t think of, we can’t even imagine.

      It’s funny. If you go back and you look at all the ads for mobile phones, like, for 10 years, Nokia and all these folks, they were all trying to convince people to use mobile phones and there’s always stocks, weather, email. There’s literally I think no person in the history of that field that predicted, you know, half the things that we’re using today. So, for me, I think that framework kind of applies whenever there’s a really big breakthrough technology. It just takes a long time to really explore the new design space it was on.

      Sep: And, you know, I think one of the reasons for that is a lot of times, the things that are new arise from the things that are old just at scale, at quantity, you know? And that’s actually really interesting because it helps give a framework for predicting things. So you could imagine, for example, blogs were predictable from zines before the internet, you know? But it would be qualitatively different because then you imagine what happens if there’s like thousands and thousands of zines and anybody could access those zines and so on?

      And so, then, that kind of starts the creative process going.

      Chris: And then, I’ve been directly involved in this, in the infrastructure stuff, people were working on it but it was frankly a little academic until recently. And so, the fact that the applications have taken off so much, and it’s made this scaling problem like a really, really urgent issue. I think it will dramatically accelerate the pace of innovation on the infrastructure side, right? It’s no longer academic, it’s now a very practical problem and there’s real customers and people willing to pay money.

      And, you know, the same feedback with…you’ve seen, I think, throughout the history of computing where the app developers on the first iPhone start pushing it to the limit. And that pushes Apple to, you know, go faster and the chip guys to go faster and the whole thing. And then you get that beautiful flywheel that drives everything forward.

      Sep: And this is something that’s been very much on our minds as we’ve been developing Celo. So, basically, kind of when we started Celo, the conversation that we were having was, the blockchain reminded us that money is just a technology. And, of course, money has always been a technology. It’s just hard to remember that it’s a technology because its features haven’t changed very much for the past 300 years. But as a technology, its features can change and as a widely used technology, its features have an impact on the society that uses them.

      So, I remember when the internet was first getting popular, people were like, “Whoa, you could imagine putting the whole encyclopedia on the internet.” And that was true, but it underestimated the true potentiality of the internet, which was that the encyclopedia would be part of a much richer, much bigger information ecology.

      And so, similarly, I see the same thing happening in money, in value. National currencies will continue to exist and continue to be important, but there will also be local currencies, regional currencies, global reference currencies, store-of-value currencies, medium-of-exchange currencies, functional currencies, all interoperating with one another in a rich ecology, not dissimilar to the internet.

      We now are starting to have the technology to implement these ideas at scale. But to do a number of these things right, we needed some form of stabilization of cryptocurrency. We needed some methods around identity. We needed advances in light client, and so on. And so, that helped guide the infrastructure that we’re building to enable this.

      Chris: It’s going to be an exciting year in crypto. Celo has launched and is continuing to roll things out and a whole bunch of other exciting crypto projects. And so, sort of all of the things that were kind of hatched back in 2017, in ’16, ’17, ’18, are kind of finally all coming out now and it should be really exciting.

      Elad: It just seems like that next wave is starting up again too in terms of incrementally new things. Like YFI (Yearn Finance) I feel like just came out of nowhere, for me at least. And so I think that we’re going to see renewed enthusiasm, I think, in crypto in the reasonable near term.

      AI, GPT-3, and ‘hyper-evolutionary’ new forms

      Chris: Let’s talk a little about AI, sort of the other…it’s amazing right now. I feel like any one of these things would be a major tech trend and we have all of them going on at the same time. So, AI, I don’t personally work on it as a day job but follow it, I guess, as a hobbyist. The big news being GPT-3, which is an algorithm out of OpenAI, which has just shown kind of remarkable results with natural language processing.

      And from what we can tell, this is not going to be slowing down. Today the more computers you throw at these kind of neural networks, the smarter they get. And at least at the moment, these systems continue to scale at a pretty healthy rate. So we should see kind of more and more really interesting stuff. Elad, I think you’ve followed this area pretty closely. How are you feeling about it?

      Elad: I think GPT-3 is almost like the starting shot for a whole new interesting era in natural language processing or natural language understanding that’s going to take a decade to play out. And I think the historical antecedents or analogs are, back in 2012, there was something known as AlexNet from this guy, Ilya Krzyzewski, which was really the starting shot for machine vision in terms of a shift where that was the first time where you really saw a big step up in performance for a while and that’s really led to everything from face recognition on the iPhone to machine vision in pharma.

      Similarly, in 2013, Google switched to recurrent neural networks for speech recognition and then later really did a lot of interesting things in deep reinforcement learning. And that ended up becoming a multi-year precursor to what became things like Amazon Alexa or Echo or a lot of the really good speech recognition technologies we have. And now in 2020, I think similarly, GPT-3 is a natural language analog to these two other key moments in machine learning-based understanding of vision, speech, and now natural language.

      I actually think this may be one of the biggest shifts because if you think of how much of the world’s information is embedded in text or how much we communicate in text, this is really the big revolution. And that includes things like enterprise document processing. If you move to natural language, you can start thinking about smart data entry. All the robotic process automation suddenly becomes automated. You can effectively have APIs, in some sense, almost self-construct on top of text in really interesting ways.

      There’s things that are very tactical. For example, in your email inbox, all the replies should be auto-generated and then you should just be able to go through and approve them as a person. We’re not there yet, again. It’s a 10-year journey. But, you know, we’ll see things like that. We’ll see legal documents just auto-marked up relative to what your company would normally do. Companies like Clarity are working on early versions of that. If you’re an author and have writer’s block, maybe automatically, you get prompted for three or four different next paragraphs to kick off how you should think about it.

      Or in the long run, maybe there’s a whole class of auto fan fiction. So, you know, you really love the novel “Twilight,” and 100 different versions of “Twilight” are spawned. So you don’t have to wait for somebody to come up with “Fifty Shades of Grey.” It just auto-generates, you know, multiple different interesting, you know, fanfic stuff.

      On the gaming side, I think you’ll have non-player roles, NPCs, that seem like real people. In health care, maybe you have a mental health specialist who’s really just a robot.

      I think this is a really exciting shift and it’s going to take a long time to play out but the technology is finally starting to show hints. Just like in 2012, AlexNet showed hints of what could happen in machine vision. And in 2013, Google showed what could start happening in speech recognition. It feels like this is one of those steps. And so I think it’s significant in terms of a starting shot, although, I think it’s going to take a lot of time to play out.

      Sep: I’m really excited about the translation opportunities, in particular the opportunities to translate English to machine understandable code.

      Chris: They’ve actually had demos of this with GPT-3, right? Where you describe something and it would actually write the code for you. I haven’t personally tried it but it seems like they’re not canned demos. It really does kind of work.

      Sep: And, you know, it’s really straightforward to do that in the context of data structures. You could imagine translating a sentence into a data structure. And it’s not a far step from doing that to natural language querying of SQL. And then it’s not a far step from that to auto-generating code. And so, that’s super exciting to me because you can imagine, there are certain things that are straightforward to build if you know how to program. And they should be straightforward to build if you don’t but it takes kind of ad hoc interfacing to do. Creating a new ERC-20 token, for example, is a pretty straightforward programming task that I can see that someone could use machine translation using any of these technologies, but GPT-3 in particular, to start translating human text to machine text.

      Elad: To your point, I think the second that machines can really write and edit code and can spawn instances of themselves and self-replicate, at that point, I think we’re really shifting from a technology to a life form. And I think at that point, you know, we really have this hyper-evolutionary new form of life that’s self-replicating, self-editing. And, you know, one of the interesting things is people always think that a true AGI or self-intelligent agent will come out of Google or Facebook or one of the major companies.

      To put these threads together, one could argue maybe where it’s going to really emerge is on the blockchain where you have these really interesting human incentives and competition around something of real value. So you have sort of an optimization metric that’s very crisp when you’re competing to effectively complete financial transactions or contracts and they’re going to get more and more complicated. And so, I think the merger of these two areas will someday happen and it’s going to be fascinating to watch in terms of whether you have this sort of emergent system of self-replicating, self-editing code with strong financial incentives built into it.

      If you look at the biology side of things, that replication plus mutability plus selection is really what drove the emergence of intelligence, right? And so, really, the selective function is you need to have a large enough number of different beings or entities. You need them to be able to change at some rate so that they start adapting to their environment they’re being selected for, and then you need that selective pressure.

      And when you start having machines be able to edit themselves and to write themselves and replicate themselves at scale, you’re both expanding the number of potential entities that are evolving. But you’re also upping the clock rate. You’re not waiting for a person to write something and test it and then iterate on it and then test it again and try and understand it and theorize and then write more code. You just have systems that are replicating and changing themselves.

      And imagine if as a human, you could edit your own DNA and change certain features and experiment with that very rapidly. That’s what’s going to happen in the world of code. And so, I think it’s a long time away, but once code can write itself, I think that’s really when things kick-off for the emergence of a true AGI-based life form.

      Chris: There’s no reason this couldn’t be applied for any kind of symbolic system. So for a mathematician, you know, the computer suggests five different proofs. Scientists, the computers suggests five different theories or interpretations or models or whatever it might be. And maybe in the near term, it works alongside a human. Maybe at some point, the machine gets so good, it doesn’t need that. And it’s probably, going back to the framework I was suggesting earlier, the stuff we’re describing falls in the category of doing existing things better. There will probably be crazy new things that we can’t even imagine right now but some developer or entrepreneur will come up with.

      Elad: The analog I’ve heard or the analogy I’ve heard for GPT-3 is it’s kind of the clever student who didn’t really study for the exam and half the time kind of bullshits it and half the time knows it. And to your point, GPT-3 can write the next paragraph. The question is, what does GPT-20 look like? GPT-50? As we iterate on these systems, you suddenly have the thing that can really write the fan fiction novel for “Twilight.” So it’ll be really fascinating to watch.

      Full-stack startups

      Chris: So the last thing we’re going to talk about is what some people call full-stack startups, which is sort of a new way to build startups. Sep, it’s a concept you were interested in. But essentially, the idea is whereas in the old days, software startups mostly stuck to just building software, more and more entrepreneurs are building companies that are sort of software-enabled but also build core capabilities in other areas.

      So, just as an example, in fintech, it used to be that the only way you would go to market, you’d build software and try to sell it to a bank or an existing financial institution. Now, more and more, you have these things, like Chime is an example, of an online bank that just sort of bypasses Citibank and goes directly to consumers. It’s an app. You can download it.

      Robinhood is another good example. Instead of building software and selling it to Schwab, they just built software and built an app and went directly to the public, right? And this is happening as sort of a new design pattern for startup organizations that’s, I think, letting startups penetrate more and more deeply into industries that had previously kind of resisted software innovation. Sep, I know it’s a topic you’re interested in.

      Sep: For the audience, Chris wrote a blog post called “Full-Stack Startups” back in 2014, which is, I’d say, a must-read. It’s one of the most concise and articulate descriptions of this phenomenon that I’ve ever read. I think, basically, kind of full-stack startups were later to emerge than pure software startups for a variety of reasons. Mostly because there was a fair amount of low-hanging fruit in software itself and it is harder to do a full-stack startup because you basically have to start two companies at the same time.

      I mean, if you’re starting a full-stack construction company, you have to start a construction company and a software company at the same time. And it’s hard enough just to start either. And full disclosure, I am a co-founder of Mosaic, which is a full-stack construction company. So I’m biased here. But once you’re able to do that, if you’re able to do that, then it allows something really powerful, which is it allows you to write software not just for existing processes, but it allows you to innovate on process at the same time as you innovate on software. And very specifically, it allows you to innovate on process in the way that software enables.

      And so, in the same way for crypto, software increases the expressive range of a whole range of things. And that expressive range allows new processes for things like building houses or selling eyeglasses or so on. And it’s really helpful to be able to have a really tight loop between changing the process itself, which is not inherently software-based but new software allows you to do that and then to iterate on the software itself. And so, that kind of opens up an area of innovation that is really difficult to do with either side of the stack alone.

      Elad: It seems like there’s a lot of other places where that approach that you mentioned, Sep, really applies. I mean, a company I co-founded that, you know, I haven’t really been operationally involved with for many years is Color Genomics and it’s doing a large proportion of COVID testing in a number of different markets. And a lot of the value, I think, of what the company does on top of just running a vertically integrated lab and other software around it, is all the virtualized care delivery and all the patient interactions, doctor interactions, etc., beyond just, “Hey, can you run a better lab?”

      And so I think, to your point, that vertical integration has made a huge difference for Color as an example. And similarly in real estate, not just Mosaic, but it seems like Opendoor, which is literally going in and repainting the interiors of houses as it buys them and things like that and is layering on mortgage and title and everything else of the home purchasing process. It seems like there’s just an enormous amount of innovation in terms of the ability to build something that’s full-stack.

      Sep: You know, I mean, it’s interesting. I’ve had a similar question, a little more general from, Chris, the time you wrote that blog post, which was…”Why is there not a flowering of full-stack companies in the same way that there’s a flowering whenever there’s a clear possibility of innovation?” You’re not seeing as many full-stack startups as you are seeing with crypto, for example. I’ve come to two reasons. I imagine there’s more.

      The first is that it is really difficult in either context, either in the startup context because it requires kind of an expertise in two very disparate areas as a startup. And I think the second thing is I think it’s tantalizing to take a big industry and, say, dabble on some technology and it will become a full-stack company. But I think you have to have a specific point of view around what the technology is and a real innovation in that technology. And so, I think for those two reasons, I think it’s just like the rewards are great but the difficulty is hard.

      Chris: Netflix is a really interesting example, right? So Reed Hastings, obviously a genius, but his prior company to Netflix was Purify, which is a debugger. I mean, it was a very, very technical product. He’s very much a computer scientist, you know? And then, he built…you know, now Netflix is doing all this original content, is becoming more and more dominant in the movie industry. I don’t know…you know, this pattern of having a technologist figure out the other industry, in that case, Hollywood, like, I don’t know if there are examples of the opposite happening of the Hollywood people figuring out the technologies part, you know?

      Sep: Yeah. And there’s few examples of Netflix too.

      Chris: Sure. And it’s so hard. And that company… I mean, that’s an amazing story, just independently. They had to pivot multiple times while being public. So, you know, and there’s just so few people…

      Sep: Right. And he’s a remarkable entrepreneur.

      Chris: Yeah. Maybe they’re just so few people like him and Elon Musk and it’s such a hard thing to do. And, you know, raising…it requires a ton of capital and decades of work. So, I think that might be why there’s not more. It’s just really hard.

      Sep: For us, the way we were able to do it is Salman, my co-founder, he has a Ph.D. in computer science from MIT but he also grew up in a construction family. So he had deep expertise in construction and deep expertise in computer science both from a young age.

      Elad: I think it’s kind of notable or interesting that most of the examples I can think of of really successful full-stack startups are second-time founders. So, with Mosaic, you know, your co-founder, obviously, had an amazing background in terms of family construction and everything else. But the flip of it is you, Sep, had already started, you know, companies before. Elon Musk had two successful outcomes before. Reed Hastings had a successful outcome before. So, it almost feels like you need a stable financial base plus enough know-how in terms of building a company to begin with so that you can take on this extra challenge of doing a second piece of it, you know, to Sep’s point around needing to build two companies at once.

      Chris: That’s a good point. Thanks, Sep and Elad. That was awesome.

      • Elad Gil

      • Sep Kamvar

      • Chris Dixon is a general partner at a16z, where he leads the crypto/ web3 funds. Previously, Chris was cofounder & CEO of startups SiteAdvisor and Hunch (acquired by eBay); and an early blogger at cdixon.org.

      How to Decide, Convey vs. Convince, & More

      Annie Duke and Jeff Jordan

      It seems like investors are especially obsessed with the psychology of decision making — high stakes, after all — but all kinds of decisions, whether in life or business — like dating, product management, what to eat or watch on Netflix — are an “investment portfolio” of decisions… even if you sometimes feel like you’re making one big decision at a time (like, say, marriage or what product to develop next or who to hire).

      Obviously, not all decisions are equal; in fact, sometimes we don’t even have to spend any time deciding. So how do we know which decisions to apply a robust decision process too, which ones not to? What are the strategies, mindsets, tools to help us decide? How can we operationalize a good decision process and decision hygiene into our teams and organizations? After all, we’re tribal creatures — our opinions are infectious (for better and for worse) — so how do we convey vs. convince, and not necessarily agree but inform to decide? Especially given common pitfalls (resulting, hindsight bias, etc.), and “the paradox of experience”, including even (and more so) winning vs. losing.

      Decision expert (and leading poker player) Annie Duke comes back on the a16z Podcast — after our first conversation with her for Thinking in Bets, which focused mainly on WHY our decision making gets so frustrated — to talk about her new book, which picks up where the last left off, on HOW to Decide: Simple Tools for Better Choices. In conversation with a16z managing partner Jeff Jordan (and former CEO of OpenTable and former GM of eBay among other things) — so, from all sides of investing, operating, life — Annie shares tips for decision makers of all kinds making decisions under uncertainty… really, all of us.

      Show Notes

      • Cognitive biases that affect decision-making [3:05] and tools for overcoming them [6:08]
      • Disadvantages of the traditional “pros vs. cons” list [11:44]
      • How long decisions should take [14:43], and how decision “hygiene” can streamline the process [20:14]
      • Making decisions within groups [24:47] and shortening feedback loops [31:14]
      • Considering optionality reversibility [36:08] and hedging bets through decision-stacking [40:31]

      Transcript

      Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today, we have another one of our early book launch episodes for a new book coming out next week, by expert decision strategist — and leading World Series of Poker player — Annie Duke. You can catch the podcast we did with her a couple years ago, for the paperback release of her first book, “Thinking in Bets.” That episode was with me and Mark interviewing Annie, and was titled “Innovating in Bets” — as is perhaps also one of the signature themes of this podcast. But in this episode, we talk about her new book, “How to Decide,” which picks up where the last book left off. And the discussion that follows covers lots of useful strategies, tools, and mindsets for helping all kinds of people and organizations decide under conditions of uncertainty.

      Annie is interviewed by a16z managing partner, Jeff Jordan, who was previously CEO and then executive chairman of OpenTable, former GM of eBay North America, and much more. They begin by quickly covering common pitfalls in decision-making, then share specific tools not to do and to do, including how to operationalize good decision hygiene into teams. And when to spend time deciding or not, especially when not all decisions are equal and some may seem bigger and more impactful — whether it’s investing in life decisions, like getting married, or business decisions, such as what product to invest in, or what strategy to pursue, or what market or what investment. As a reminder, none of the following is investment advice, nor is it a solicitation for investment in any of our funds. Please be sure to read a16z.com/disclosures for more important information.

      Jeff: So, Annie, as the author of one of my favorite books, what motivated you to do a sequel — your new book, “How to Decide: Simple Tools for Making Better Choices”? “How to Decide”?

      Annie: So, when I think about what “Thinking in Bets” was about, it was really the way that our decision-making gets frustrated by this, kind of, discorrelation between decision quality and outcome quality. And then toward the end of that book, I was — kind of, a little bit of an exploration about how you might become a better decision maker given the uncertainty, but it was mostly a “why” book. And so, this is really trying to lay out for people — how do you actually create a really solid and high-quality decision process that’s going to do two things? One is, get a pretty good view on the luck, which you need to do. You need to be able to see it for what it is. Obviously, you can’t control it, but you can see it. 

      But then the other thing, and I think this was something that was really fun — I got to really dig deep into this problem of hidden information, that when we’re making decisions, we just don’t know a lot, because we’re not omniscient. And we also aren’t time travelers. And so, I got to actually do this really deep exploration into how you might actually, really, improve the quality of the beliefs that you have, that are going to inform your decisions. Which was a topic I covered a tiny bit in “Thinking in Bets,” but here we do like a super deep dive.

      Jeff: It is a super deep dive. And why I love your books is — it’s so germane to what we do in our day job, which is [to] make decisions under extreme uncertainty. So, to recap, [talk about] why trying to learn from experience can go sideways.

      Common decision-making pitfalls

      Annie: Sure. So, you know, both of my books, kind of, start a little bit in the same place and then they diverge from each other. But I think that’s because it’s the most important place to start. What I talk about, at the beginning of this book, is what I call the paradox of experience — which is, obviously — experience is necessary in order to become a better decision maker. You do stuff, the world gives you feedback. You do more stuff, the world gives you feedback. Hopefully, along the way, you’re becoming a better decision maker, given that feedback.

      The problem is that any individual experience that we might have can actually frustrate that process. We can learn some pretty bad lessons when we take any individual piece of feedback that we might get. So experience, while necessary for learning, also is one of the main ingredients that makes us worse decision makers. And it really just, kind of, comes from this problem, that in the aggregate, if you get 10,000 coin flips, we can say something spectacular about the quality of our decisions and what we should learn or not learn from them.

      But that’s not the way that our brains process information. Our brains process information sequentially, one at a time. And because we’re, sort of, getting these outcomes one at a time and we’re just taking really big lessons from something that’s really just one data point. And the two main ways that that frustration occurs is because of resulting, which obviously I cover quite a bit in “Thinking in Bets,” where we use the quality of an outcome to derive the quality of the underlying decision. You can run red lights and get through just fine. And you can run green lights and get in accidents. So, these things actually are correlated at one, but with resulting, we act like they are. And then the other problem is hindsight bias. We aren’t really good at, sort of, reconstructing our state of knowledge at the time that we made a decision. And so, once we know the outcome, you not only kind of view that as inevitable, but we’ll also, sort of, think we knew that that outcome was going to occur — none of which are true. So, those two things combined are really problematic.

      Jeff: You had this beautiful imagery of decision forestry, which resonated with me.

      Annie: I sort of think of them as cognitive illusions. What those illusions are creating for us is the idea to say — it’s the only outcome that could have occurred. In reality, though, what we know is that, at the moment that we make a decision, there’s all sorts of different ways that the future could turn out. When we’re at the moment of a decision, we can see all those branches of the tree, where I become a fireperson, or I become a poker player, or an academic, or whatever, you know — sort of, imagining all the different ways that the future could unfold. But then after the future unfolds as it does, we take a cognitive chainsaw to that whole tree, and we just start to lop off the branches that we happen not to observe. In other words, we, sort of, forget about all the counterfactual worlds. And we end up thinking that there was only this — only this one branch that could have happened, because we sort of chainsawed everything else away. We sort of forget that there were other paths that could have occurred.

      Jeff: How do you keep the forestry from lopping off the branches? As you started turning to how, you started with some really useful tools.

      Tools for analyzing options

      Annie: So, there’s two tools that you could do when you’re thinking about that, actually three. The first has to do with trying to reconstruct what the actual state of knowledge that you were in. When you think about — what did I know beforehand, and what did I know afterwards, you can now start to sort of reasonably see — what was the information that was informing the decision at the time? When you actually go through this process, you’ll spot, “No, wait a minute, that was something that revealed itself after the fact.” That’s one thing that can be very helpful.

      Another thing that can be very helpful is to actually go through this process of thinking about this two-by-two matrix of the relationship between decision quality and outcome quality. So, there’s a quadrant, which is — good decision, good outcome, which you can think of as like an earned reward. Good decision, bad outcome — that would be bad luck. Bad decision, good outcome — dumb luck. Bad decision, bad outcome — I guess would be, like, your just desserts. When you’re thinking about any outcome that you’ve had in your life, if you do that over time, what you’re going to see is that you’re going to have certain patterns about which quadrant you’re really filling in a lot. So, if you’re seeing that you’re really only putting things into, like — good, good, bad, bad — you need to start seeing how luck is influencing you.

      And then the other thing you want to do is just start thinking about particularly the good, good quadrant. Because we are asymmetrically willing to go and try to find some luck in there. Let me explain what I mean. So, if you have a bad outcome, you already feel bad. You’re sad because you lost. And it’s, kind of, nice to go in and deconstruct that, and analyze process, and really look at the quality of the decision that led to that outcome. Because if you find some bad luck in there, you get a little relief.

      Jeff: You, kind of, get off the hook.

      Annie: Right. It’s like a door out of the room. Luck is giving me a way out of this. So, we’re actually pretty eager to go around and explore those bad outcomes. What we’re not so eager to do, though, is when we have a good outcome — to apply the exact same process. To actually spend some time in there thinking about, “Well, you know, what was luck or was there a better way?” And the reason why we don’t want to look at that is because we feel pretty good. If you find out that you won because of luck, that’s a door that you actually don’t want to have open for you. 

      So, I actually put a lot of focus, when I’m thinking about using this tool, of really digging into that one quadrant. And what you can see is, in order to actually be thinking about which quadrant that fits in, you have to actually apply this other tool — which you can do in retrospect — which is actually to do some exploration of, like, what are the other things that could have happened? Because if you don’t understand those counterfactuals, it’s very hard to actually appropriately place any outcome into the right quadrant. So, I have tools in the book which will help you, sort of, reconstruct these things retroactively.

      Jeff: It’s kind of interesting. The investment community often tries to capture the thinking at the time through the investment memo. Which then, you know, records, okay, you know — these are the potential outcomes that we can envision, here are the probabilities of the different outcomes. And in total, we’re willing to make this bet, even though there are some outcomes that are pretty unattractive, to say the least.

      Annie: And that, absolutely — if you think about a knowledge tracker, that’s what you’re doing. It’s like you’re trying to reconstruct an investment memo. It’s better than nothing. But what you really, kind of, want to be doing is — doing this stuff prospectively. You want to have some sort of record of not only what you thought at the time, but also exactly what you said. Like, what are the ways that we think this could turn out? Like, what are the payoffs of each of those possibilities? How probable do we think those are? So, you can actually look at, generally, two things — what’s the expected value, what’s my downside risk. And then you can, obviously, compare options to each other. What I think is actually really important, though, about thinking about this, like, evidentiary record — that you’d like to create at the time of the decision, as opposed to [trying] to reconstruct, is that it’s not actually an extra step.

      Like, people talk about decision journals, which feels like work. Because it feels like an extra step where you’ve done the decision, and now you’re trying to record everything. The fact is that a really great decision process is going to produce this evidentiary record naturally. And obviously we’d prefer to have that, because what the evidentiary record is giving you — what that investment memo is supposed to give you — is, sort of, what your expectations of the world are. Not just like, do I think I’m going to win or lose at this probability, but also, like, what do we think is going to be true with the world in general? What I find in my work is that when people lose, they’ll do these process dives. The problem is when there’s a big win, they’re like, “We won.”

      Jeff: Yep, exactly. When an investment goes bad, you do spend time trying to say, “Okay. What can I learn? What can I do differently?” And then when it goes well, you just spike the ball in the end zone and do a dance.

      Annie: And we really are just, like, spiking the ball. But there’s so much to be learned from the wins as well, and I would argue, actually, more. Particularly, by the way, when the <inaudible>, it’s like, there’s going to be — in a lot of ways, there’s more to be learned from the wins than the losses, right? Because the thing is, like, you know, you can win for all sorts of reasons that you didn’t expect. And yet we spend a lot more time in our decision process, exploring the losses that were for reasons that we expected — than the wins that might’ve been reasons that were unexpected. Maybe we could have cleaned up the process, or there was information that we’re missing that we could have applied, so on and so forth. We’re, kind of, losing a lot of the learning time. We’re not being very efficient when we do that. And the other problem with that is actually that that has downstream effects that are quite bad. I’m going to do things that are very consensus. So, I’m going to want everybody to agree with me.

      Jeff: Yeah. That resonates a lot. So, you take on using a pro-con sheet. And it was funny. I was cleaning up offices a couple years ago, and I found sheets in different places, and aggregated my career decisions. And, you know, I came to the conclusion that they were pretty much worthless. And so, you come to the same conclusion in the book. Why are pro-con sheets worthless?

      Annie: So, let me just say, a pro-con list is actually a decision tool. And if you have a choice between that and nothing, I think a pro-con list is very slightly better than nothing. But here are the problems with the pro-con list. The first is that it’s flat. It lacks any dimension. It’s like a side-by-side list — here are the pros, here are the cons. And I don’t really understand how you would weigh one side against the other without adding some dimension to that list. And that dimension would be two things. One is, how bad? What’s the magnitude? The other dimension that’s missing, which is terrible, is probability. 

      So, in that sense, I’d rather just use the decision tree. And for an option that I’m considering, I want to just think about what are the reasonable possibilities, what are the payoffs for those, and what are the probabilities of those things occurring? And then I can add that dimension back in. Without that dimension, it’s not a great tool for comparing one option to another — because, again, I can’t calculate, like — any kind of weighted average here. If, like, I’m choosing between two colleges, is the one with more pros, like — am I supposed to not go there? I really, kind of, don’t know, because I don’t have this dimension.

      And then the third problem, which I think is actually the most dire, is that what we’re really trying to do is to reduce the effect of cognitive bias. Pros and cons lists actually amplify all of that stuff. It’s, kind of, a tool of the inside view. And let me just say for people listening — I imagine some people saying, “No, when I go to make a pros and cons list, I haven’t decided yet.” I have news for you. The minute you start thinking about a problem, you’ve already started deciding. You know, regardless of whether you’ve made that explicit or not, you’ve already started to get yourself to a conclusion. And now when you go to do a pros and cons list, it’s just going to amplify the conclusion that you already want to get to. So, I think it’s just not a very good tool.

      Jeff: My worst career decision, by a mile, was joining a company called real.com, right at the beginning of the internet era. It was being purchased by Hollywood Entertainment, which ran the Hollywood Video stores. And it was a bad decision. I <inaudible> in a year, I got scars. But when I went and saw the pros and cons, the pros were aspirational and the cons were delusional. I clearly had decided before I started the list.

      Annie: Yes, exactly. When we start to use something that feels objective, like a pros and cons list, we get that feeling of like — well, now I can have confidence that it’s a really good decision. So, one of the things that I’m very wary of — is that I think that there’s certain things that can come into a decision process that feel like it’s certifying the process. So, we end up with this combo of a decision that isn’t really better, but that we feel is much more certified.

      How long a decision should take

      Jeff: I love the tools you described using the decision trees. The prospective gathering of information. Then you took your “how” into an interesting direction. And I really enjoyed the part on spending your decision time wisely. <Oh.> So, it’s a book about, you know, making great decisions — and then you start talking about all the decisions that you shouldn’t apply it to.

      Annie: <laughter> I know. So, I spend the first six chapters really, kind of, laying out what a pretty robust decision process would look like. And then I, sort of, take a hard left and I say, okay — so now that you know, mostly you shouldn’t be doing that. Which I know sounds a little bit odd, but it’s this meta skill of understanding that obviously you can’t take infinite time to make decisions, because opportunities expire, and you’re losing the ability to do stuff in between. And so, we want to really think about what types of decisions merit taking time, and what types of decisions merit going fast. And it just turns out that most of the decisions that you’re going to make on a daily basis are ones that you should be going fast on — much faster than you actually do. And in some ways, I think that people, sort of, have it reversed.

      Jeff: Throw out a couple examples, because that’s where it really came alive to me.

      Annie: Okay. So, let me ask you this. What’s your guess — obviously pre-pandemic — what’s your guess on the average amount of time that an adult in America takes on — what to watch on Netflix, what to wear each day — I mean, at the moment it’s sweatpants, but, you know, we’ll ignore that — and what to eat?

      Jeff: If you’re my mother-in-law, she used to spend a half hour every time we went to a restaurant.

      Annie: So, like, she’s not even that much of an outlier. If you add it all up, over the course of a year, the average adult is spending between six and seven work weeks — like, literally, on just those three decisions. I’m sure she’s looking at the menu, and then it’s quizzing like all the waitstaff, and asking everybody else at the table what they’re going to order — like, try to go back to the chef, looking on Yelp. So, here’s my question for you. Let’s say that we ate a meal together, and you were trying to decide between two dishes. Like, what are two dishes that you would have a hard time deciding between?

      Jeff: Fish and a good veggie stew.

      Annie: Okay, okay. So, you’re trying to decide between those two things. If you’re [your] mother-in-law, you’re quizzing everybody. So, let’s imagine that you ordered the veggie stew and it came back. And let’s imagine you got this bad outcome, where the food was really yucky and you didn’t even finish it because it was so gross. So, now let’s imagine it’s a year later and I say, “Hey, Jeff, how are you feeling right now, happy or sad? So, you remember that horrible veggie stew you had a year ago, how much of an impact did it have on your happiness today?”

      Jeff: Zero.

      Annie: Zero. Okay. So, let’s imagine I catch up with you in a month and I say, “Hey, Jeff, feeling happy or sad right now? Do you remember that horrible veggie stew you had like a month ago? How much of an effect on your happiness did it have today?”

      Jeff: None.

      Annie: None. What if I catch you a week later, by the way?

      Jeff: None. Now, if it had been the fish that had been bad, like, a week maybe…

      Annie: Maybe, but not the veggie stew. <laughter> Okay. So, what I just walked through with you is something I call the happiness test. I use happiness, generally, as just a proxy for, are you reaching your goals? Because we’re generally happier when we’re reaching our goals. So, you can substitute any goal that you have in there. And this is a way for us to figure out how fast we could go. Because, basically, the shorter the amount of time in which your answer to the question is — “Did it affect your happiness at all?” — is no, the faster you can go. Why? Because there’s a tradeoff between time and accuracy.

      So, in general — not always, but in general — the more time we take with a decision — and there’s more time for us to, like, map these things out, and actually calculate, like, expected values, and figure out what the volatility might be. Or gather information, get more data, all of those things. Generally with time, we should be increasing our accuracy. So that’s why we can speed up — I’m assuming no food poisoning here — that when we look at the worst of those outcomes, that it has no effect, it’s neither here nor there. Which means that — we can take on the risk of saying, “I’m going to spend less time, because I’m willing to risk the fact that I might increase the probability of the worst outcomes, because it doesn’t really matter to me.”

      Jeff: Then you make another point — that you can repeat the decision, next day at the restaurant — and order the fish instead of the tasteless stew.

      Annie: That’s the other thing that you can look at, which is — when you have these low-impact decisions that are quickly cycling, and they repeat very quickly — so that’s, like, what to watch on Netflix, what to wear, what to order at a restaurant — we should go really fast for two reasons. One is you’re going to get another crack at it in like four hours. And then the other is that — one of the things that we actually don’t know well, although we think we do — is, like, our own preferences. We’ve all had that experience of having a goal, achieving it, and realizing that wasn’t really what we wanted in the first place. And then there are certain types of decisions where it’s just really helpful to, sort of, get some feedback from the world.

      So, when we can actually cycle these decisions really quickly — and I’m not really too worried about, like, making sure I’m making the best possible decision in terms of accuracy. What I’d rather do is get a lot of cracks — get a lot of at-bats — so that the world can start giving me information back more quickly, and I can start cycling that feedback a lot faster. Then I’m going to build much better models of the world. And what my own preferences are, and what my own goal goals are, and what my own values are, and what works and doesn’t work. Such that when I do actually make a decision that really matters, my models of the world are going to be more accurate — by having just, sort of, like, done a whole bunch of stuff really fast and not really cared whether I won or lost.

      Practicing decision “hygiene”

      Jeff: That makes perfect sense. Now, one of the chapters that I loved was decision hygiene. I found this book fascinating from the perspective — as both an investor and a former operator. I mean, an investor, it’s obvious — you’re making two or three investments a year. You’re seeing, you know, hundreds of companies. How do you decide? But as an operator, there are a few decisions you make each year that are super, super important. Particularly, the ones that I used to labor over was — okay, you have to commit. You have to invest your product resources — your most valuable asset, your engineers — into specific deliverables. You know, is it going to be A, B, or C? And that’s the most important decision I made all year, other than possibly people decisions. Explain a little bit — how you can maintain great hygiene. It resonated in both my professional experiences in a really significant way.

      Annie: I have to say, like, the decision hygiene stuff — and the ideas of predicting these intermediating states of the world — apply so much in a startup environment. Because obviously, kind of, the nature of a startup is that you do have very little information, and you’re making pretty big bets on a future that, by definition, is going to be somewhat contrarian. So, making sure that you don’t get in this, kind of, group think. Like, I’m not saying don’t believe in yourself, of course — but this is actually a way to have more belief in yourself. Because the quality of the decisions that are going to come out of a good decision hygiene process are going to be so much better. And that becomes much more important in a situation where we are at a paucity of information. And then it starts to actually close feedback loops more quickly for you, which also increases the quality of your models and information. So, I actually can’t think of a place where this is more important than in a startup environment.

      So, let me just start, kind of, the premise — why you need some decision hygiene. I don’t have control over luck. What I can do is, I can make decisions that reduce the probability of a bad outcome. You know, even if I make a decision that’s only going to have a bad outcome 5% of the time, I shall observe it 5% of the time. And luck is what is determining when I observe that bad outcome. So that’s kind of one side of the puzzle. The other side of the puzzle has to do with how you construct your decision process. What do you think your goals are? What do you think your options are? What do you think your resources are? What do you think those possibilities are for any given option you’re considering? What do you think the probabilities of those things occurring are?

      Basically, your whole process is built on this foundation. Like, that whole house is sitting on top of a foundation — which is your beliefs. And by beliefs, I don’t mean things like religious beliefs. I mean, just like — what are your models of the world? How do you think the world operates? What are the facts that you have? What’s the knowledge that you have? And that foundation that that whole process is sitting on has two problems. One is that a bunch of the things we believe are inaccurate, so it’s like cracks in the foundation. And the other is that we don’t know very much. So, it’s like a flimsy foundation. The solutions to both problems are the same — which is that we need to start to explore that universe of stuff that we don’t know. That’s where we run into new information that helps us beef up our foundation. And it’s also where we happen to run into corrective information —  things that can correct the inaccuracies in the things that we believe.

      The other thing that helps us, too — when we were talking before about the pros-and-cons list that gets you, kind of, caught in your own cognitive bias — is to realize that like a lot of the cure to those kinds of problems is to get other people’s perspectives. So, two people can be looking at the exact same data, and they can come to very different conclusions about the data. That’s what a market is. It’s different perspectives colliding. All right. So, having set that stage, one of the best things you can do for your decision-making is finding out what other people know and what their perspectives are on the problems that you’re considering. The problem is that without really good decision hygiene, you’re not actually going to be able to execute on that properly. So, let’s figure out — how do we get this into a team setting? Basically, human beings are very tribal, and we’d like to, sort of, agree with each other more than we actually do. And our opinions are really, actually, infectious. So, in order for you to know that you disagree with me, what is the thing that you need to know from me first?

      Jeff: Well, what do you think?

      Annie: Right, exactly. And this is where we get into this huge problem in interpersonal communication. When people ask for feedback, pretty much 100% of the time, they tell the person what they believe first. I’m thinking about a particular sales strategy or whatever. And I will lay out for you, not just the information that you need, but I also tell you my opinions on that.

      Jeff: “Give me your unbiased opinion,” right. Now that I biased you hugely, right.

      Annie: Right, exactly.

      Jeff: So, the reason your decision hygiene point, maybe, was so interesting to me — is you called out one of my tools that I used as an operator, which was quarantining in group settings. I found, at OpenTable, that if I walked in and had, you know — put a strategic choice on the table, there was one-and-a-half people in there who would drive the discussion, and their opinion would always carry the day. <Yep.> So, I developed a tool where I would pre — on very important, big-time, strategic decisions, I would ask everyone to send me their lists of prioritizations. And then I would aggregate them, and then feed that back to the group — to heighten the contradictions, essentially. The quiet person who didn’t really want to put a contrary point of view, and spar with the other person. All of a sudden, the data is on the table because they quarantined the gathering of it. And then I found the conversation was so much better than just throwing it open and having, you know, the charismatic, loquacious, opinionated person, carry the day every time.

      Annie: I could quarantine my opinion. But as soon as someone else talks, as you just so nicely put — it’s like everybody else is infected anyway. I’m just a really huge fan of pre-work. Figure out what it is that you’re trying to get feedback about, give everybody the same information. And then actually elicit those responses. Now, the more specific, the better. So, I like them to rate it, right? Give me on a scale of zero to five. Because then I can find out, like — Jeff is a four and, you know, Annie’s a two. And maybe Jeff and Annie need to have a conversation, because it turns out that there’s quite a bit of dispersion of opinion there. What this allows you to do is — first of all, it actually disciplines your decision process, because you have to think about what are the things that matter to this decision that I’m trying to elicit opinions about. And let me be clear, it’s not that I don’t think people should provide rationales. I think those are actually quite important. It’s just that they need to have something that’s much more precise. It’s like a point estimate, because I need to see where the dispersion is and then let them give the rationale.

      Jeff: I used to give a hypothetical budget. You have a million dollars. Here are the 12 ideas you can invest behind. Deploy your budget. And each person would deploy. And then all of a sudden, you’ve got something that’s really powerful. And you’ve got — oh, you loved this idea, and you hated this idea. Let’s discuss the idea.

      Annie: Right, which I love. Exactly. So, you can actually see that they disagree with each other, or see that they do agree with each other. It also makes you actually think about — what are the component parts of this decision that really matter? You can start to actually create for yourself — almost, like, a little bit like a checklist — but here are the things that we need to pay attention to, and that I actually need to get the feedback on. So, what Kahneman would call these are mediating judgments. You’re thinking about what are the mediating judgments for any broader category that you might be judging on. And that helps you to really discipline the decision process. You then bring that together in one doc, and you sort it into — here are areas of agreement, here are areas where there’s some dispersion.

      People get to read that prior to coming into the meeting. So, they’ve actually seen, sort of, the full slate now, of what the different opinions are in the group. This does really great things for your meetings. It makes them much more efficient, much more productive. You’re not surfacing all that stuff in the room, which just takes a long time. <Absolutely.> And by the way, you’re not going to surface all of it anyway, so that’s bad. 

      But the other thing is that now you can come in and you can say, here are areas where we generally agree — yay us — but let’s not talk so much about the fact that we agree. Which is what happens in a lot of meetings, where you’ll say something, Jeff, and then I’ll go, “I agree with Jeff, and let me tell you why.” And then somebody else is like, “Yes. And I have more color to add to that,” because everybody, sort of, wants credit for that idea. But we don’t care now, because we already found out we all agree.

      Yay. Yes. Right. There was this round. Cool. Right? But now it turns out that Annie thinks the earth is flat over here. Okay. So, now what are we going to do? Jeff thinks the earth’s round, Annie thinks the earth’s flat. And that’s where you really want to be focusing your time — on places where there’s dispersion. And you want to focus that time in a way where it’s not about convincing anybody of your opinion. It’s about just informing the group. And then if anybody, sort of, agrees with you, I’ll say, “Hey, you know, Sonal, you also agree that the earth is round. Is there anything you want to add to that?” So, you’ll get to say your piece. And then, “Annie, you believe the earth is flat. Is there something you didn’t understand?” Now, notice, in no way is anybody saying, “You’re wrong,” or “You haven’t thought about it this way,” or whatever. It’s, I get to tell you, “Here’s something I don’t understand.” And then we, sort of, get to the point where I say, “Okay. Explain your position.”

      There’s really amazing things that come out of that process. Thing number one is you get much more comfortable with the idea that everybody doesn’t have to agree. Number two is people have different mental models. And so, you get to expose everybody to those different perspectives, and the different facts people are bringing to the table. So, the whole group becomes more informed, which is awesome. The third thing is that the person who is conveying their position becomes more informed. Why? Because in the process of having to defend why I believe the earth is round, I discovered that I actually can’t explain that very well. So, maybe I have to go google some stuff or look it up. And there’s going to be good stuff that comes out of that, because I’m going to be more likely to actually moderate — because I’m, sort of, poking around in my knowledge a little bit.

      And then the last thing, I think, that comes out of this, that’s really good — is that once you get into this idea of “convey” versus “convince,” you realize that you don’t need to agree to decide. You need to inform to decide. And that the idea that all of you would be on equal agreement about whether you should do something or not is completely absurd, because we don’t have to — because that’s the whole point. If you thought that that was the goal, why do you have more than one person on the team?

      Jeff: Yeah. You want a diversity of opinion. <Right.> And if you don’t tease out the different opinions, then you make an inferior decision. I actually thought this was one of my management secrets. Like, you just outed it in your soon-to-be bestselling book.

      Annie: Yeah. So, actually, what’s interesting about that problem, I think that teams often act like a pros and cons list where…

      Jeff: Interesting, yeah.

      Annie: …we have the intuition that more heads are better than one. So, when we bring more heads into a decision, you have this decision that feels much more certified. But what we know is that when you allow people to make these decisions in, sort of, committee style, like in a team meeting — that the decision quality often isn’t better. And there’s lots and lots of science that shows this.

      Shortening feedback loops

      Jeff: So, one of the things in venture that is often cited as a challenge in decision-making is [that] the feedback loops can be forever. What’s your take on that — feedback loop in decision-making?

      Annie: Yeah. So, basically my take is that there’s actually no such thing as a long feedback loop. Which I know sounds weird, right? Because, obviously, you’re saying, like, we invested in a company — we find out how it exits, like, 10 years from now. Isn’t that a really long feedback loop? But the thing is, I mean — going back to this idea that when you make a decision, it’s a prediction of the future — it’s not like you’re just predicting what the exit is going to be. You’re predicting a whole bunch of intermediating states of the world. And that might be just like, for example — like, what is the arc of the ability to attract talent for this particular founder? Just, like, as an example, right? You know, obviously, is it going to fund at the next round. I mean…

      Jeff: It’s a good example, the funding management team.

      Annie: Right. If you knew for a fact that they weren’t going to be able to hire a good team, you won’t invest in them. So, it’s really good to, sort of, make predictions about those things, and make them probabilistically. Because as you’re making these types of forecasts now, over the course — in a much faster time period, you’re starting to see — when we say that there’s a 60% chance that this intermediating state of the world is going to exist, does it absolutely exist 60% of the time? 

      Because in the end, the thing that’s so important to understand is that you are saying that you’re an expert at the market that you’re investing in. So, you want to be explicit about the things — those predictions that you’re making about that market, both near term and far term — so that you don’t have to wait around 10 years. Because the thing is, you’re going to have to make another investment in between. You can’t just make the investment, wait 10 years, get your feedback, and then make another investment. And now, if you’re actually being explicit in the way that you’re thinking about those things, you can actually create much tighter feedback loops.

      Jeff: It’s just aggregating to get it into a set of milestones?

      Annie: Right. There’s no reason you can’t do that out in the world. One of the knocks that people will say about poker is, “Oh, but you get really fast feedback and so, pooh-pooh on you.” And I’m like, well, yeah — except that it’s really just a compressed version. There’s the end of the hand, which is what you’re thinking about. But in between the start of the hand and the end of the hand, there’s all sorts of predictions that I’m making in between.

      Jeff: I’ve been an investor for nine years. The feedback loop is 10. I’ve made 35, 40 decisions. If I deferred any learning to the end, it would be pretty wasteful. And there’s another psychological thing we fight — the phrase is, “Your lemons ripen first.” So, if your company goes for 10 years, there’s a pretty good probability it’s going to have a good outcome. <Right.> But the ones that die after, you know —  can’t raise the next round, can’t have the management team. That’s when your negative outcomes manifest before your positive outcomes. And psychologically, you have to manage through that.

      Annie: Yeah. So, this actually, I think, gives you a tool to be able to do that, because you have no secondary way to be right.

      Jeff: Yeah, it does.

      Annie: Like, how am I doing in terms of, like, calibrating around how likely I think this company is to fail? You know, in what ways is it going to fail? What does that actually look like? The other thing that comes from that is that — when you make yourself, sort of, break this into its component parts, when you actually force yourself to do that — I think it actually improves the knowledge that goes into it. Because you have to start thinking about — what are the things that I know, what are the things that I can find out? What are the perspectives that I could consider? What are the mental models that I could apply that will help me with this prediction? Because it is now recorded — it is part of that evidentiary record, which we’ve already said is incredibly important — that allows you to have that look back. And because you know that you’re accountable to it, I think it actually improves the accuracy of the original decision. Because it makes you be more fox-like rather than hedgehog-like, because you know that there’s going to be a look back.

      Basically, fox-like thinking is looking at the world from all sorts of different perspectives — applying lots and lots of different mental models to the same problem to try to get to your answer. And hedgehog is like — you approach the world through your one big idea. So, you could think about, like, in investing — you have like one big thesis, instead of looking at it from all sorts of different angles. Generally, what you find is that fox-like thinking is generally going to win the day. And this is something like Phil Tetlock — I’m sure a lot of people are familiar with “Superforecasting” — talks a lot about. So, apart from the fact that you can speed up the learning cycle, I think it actually improves the decision in the moment — the knowledge that at some point someone’s going to look back at it.

      Jeff: Yeah. I think that’s absolutely true, and it’s a good tool. And we may start implementing that at the firm really soon. You know, as investors, we have the — we get the benefit of being able to make a basket of decisions, you know, diversification. A lot of the people making decisions are making, like, one decision. What is the impact of optionality? How do you deal with, you know, that one decision?

      Annie: So, first of all, here’s the secret. Your decisions are a portfolio, because you make many of them in your life. And I understand, one decision — like this particular product decision. But that’s actually, kind of, like a false segregation, because you’re, kind of, working across different decisions. But I do understand that some decisions you’re making feel like they’re much higher impact. Like, when we go back to the happiness test. Obviously, like, when you’re, sort of, putting your eggs in one product basket, this is something that if it goes wrong, it’s going to have a very big effect on your ability to achieve your long-term goals. But that doesn’t mean that you can’t think about, “How can we just, sort of, move fast? And then, how would we then apply this to making a higher quality decision about something like that?”

      So, one of the things that we want to think about besides impact, when we’re considering how fast we can go, is optionality. Which is really just — if we’re on a particular route, how easy is it for us to exit? Can we get off the route? Because obviously when we choose a particular option, we’re foregoing other options. And there’s obviously opportunity costs to those — to not choosing those options. And what we’re doing is we’re saying, “This action compared to others is going to work out better for me, a higher percentage of the time, than other options that I might choose.” But we know that after you choose something, sometimes new stuff reveals itself, or the world tells you some things — that maybe this isn’t a road that you want to be on. So, then the question just is, how easy is it for me to get off the road?

      So, one of the things that we want to look at is what people call type one or type two decisions — or Jeff Bezos says two-way door, or one-way door decisions. That when you have a two-way door, when it’s easy for you to quit — and either go back and choose an option that you previously rejected, or choose a new option that you hadn’t previously considered — that we can go faster. Because really, it’s a way to mitigate the downside, right? If I’m kind of on a bad route, I can at least get off and try to figure out how to get onto another route. So that would be like going on a date — super quittable. I can leave in the middle if I want. Getting married — a little harder, less quittable, right? So, you know, taking a few classes online — much easier to, sort of, quit than, like, actually committing to a particular college. Or renting, more quittable than buying.

      Jeff: By the way, it turns out doing online classes and going to college is now the same thing.

      Annie: It is. My children will tell you that. That is so true. But the more quittable something is, the faster we can go — because when we can quit, obviously that mitigates the effect of observing the downside outcomes. The other thing we can do is actually think about portfolio theory, but for decisions that we don’t think of as investments — even though all decisions are investments. Which is, sometimes we don’t need to choose among them. So, you can date more than one person at once, right? I actually don’t need to choose between these two options. I could actually do both at once. And then I can, kind of, figure out which one’s working better. And, you know, we did this with, like, A/B testing in marketing. That happens in software development — where you’re, sort of, trying to decide between two features. And you develop them in parallel, and you test them with one set of users, and another set of users are seeing different features.

      Jeff: A number of businesses do business locally. You’ll have restaurants in San Francisco and LA — delivers groceries in San Antonio. You can charge — you can have different pricing approaches in the different markets and just learn. I mean, no one in San Antonio is going to know what you did in Montpelier, Vermont. So, try it out and you learn and learn and learn, then you go national.

      Annie: Exactly. When we can do things in parallel, obviously we’re also better off. And then the other thing is sometimes you have an option that isn’t quittable, but you can still quit it because you can negate it. So that would be, like — let’s say that I’m invested in a stock, and it’s totally illiquid — have no ability to sell it. If I could find a stock that’s perfectly negatively correlated with the stock that I own, and I buy that in an equal amount, I’ve now solved my problem. So, I’ve quit it even though it wasn’t liquid. That’s just hedging. So, if you can find something that’s, kind of, negatively correlated with the first thing, then you can actually go faster. So that, you have to think about in advance, right? This thing is pretty illiquid. It’s gonna be hard for me to exit. Is there something where if new information reveals itself, I can, kind of, just negate that decision? And if you can do that, then you can also go faster.

      Decision-stacking

      Annie: So, now that we’ve, sort of, understood, like — there’s the impact of the decision, and then we have this optionality thing — like, can you quit, can you hedge? We can now get to this idea of decision stacking. Which helps us when we have to make this big bet — is to say, what are the things that I can do before — that are going to help me to gather information? So that when I do have to make that big bet that’s going to be hard to reverse, my model of the world is going to be better. So, how can I start to use this idea of making some little low-impact decisions, just to kind of see what’s going on — to do some things in parallel? I can blunt it in order to start building better models of the world, so that when I do actually put this out into the world, then I know something more about the market.

      So, when you know that you’re going to have one of those on the horizon — I mean, they normally don’t just, like, hit you by surprise. So like, “Oh crap, I’ve got this decision to make!” It’s just really good to try to stack these other types of decisions in front of it. Because when you do actually have to make that decision tree — when you are actually trying to figure out like what the user uptake of something is going to be or, you know, whatever — what people are willing to pay for something —your model is just going to be so much stronger for having thought about what are the things that I could do in front of that really big decision.

      Jeff: De-risking. You know, trying to get all these little nuggets of directional information to give you higher confidence in the really big decisions.

      Annie: Yeah. And you can even apply this in, like, all sorts of different places. But, you know, the classic thing is dating before you marry. One of the things that I find is that when people aren’t, like, 90% sure that it’s the right path, that they’re pretty reticent to actually execute on it. But, you know, we have to make lots of decisions where we’re 60%. And by the way, when we estimate ourselves to be 60% on something, we’re overestimating that — because we’re just deciding under uncertainty. It’s just, kind of, how it is. We don’t have a lot of information. So, once you have an option that appears to be significantly better than the other ones, you just have to do a final step — which is to say to yourself, “Is there some information that I could find out that would cause me to flip this option in relation to the other options that I have under consideration?” And now it just becomes really simple. If the answer is yes, you can just say, “Can I afford to go get it?” You might not be able to because of time or money. And if the answer is yes, I can afford to go get it, go get it. If the answer is no, look — this is the state that we’re always making decisions under. I don’t have a time machine. My decision-making would be much better if I had a time machine. Sadly, I have none.

      Jeff: That’s the next book, the time machine.

      Annie: The time machine, right. I know, right, exactly.

      Jeff: This has been a fascinating session. Thank you for spending the time with us on the “a16z Podcast,” to paraphrase Sonal.

      Annie: I am so grateful to have gotten to come on and to get to discuss this stuff with you. I had so much fun.

      Jeff: I’ve been looking forward to this conversation for quite a while.

      Annie: No. I’m so excited, because I did get delayed a little bit due to a small misprint.

      Jeff: That wasn’t a small misprint. That was a big misprint. And now, I have an eBay collector’s item, which I’m the perfect person to know how to monetize.

      Annie: Yeah, right. So, for people who don’t know, is that — books get printed in, sort of, 20-page sections that get bound together. And really, a lot do with COVID — one section got printed twice, and one was totally missing.

      Jeff: I was just questioning my mental facilities while reading, because I was…

      Annie: But don’t worry. It’s been repaired.

      Jeff: Excellent.

      Annie: October 13th, when the book is out, you will get an appropriate copy.

      Jeff: That will be awesome. I can’t wait.

      • Annie Duke

      • Jeff Jordan is a managing partner at Andreessen Horowitz. He was previously CEO and then executive chairman of OpenTable.

      Fintech for Gen Z and Millennials

      Amira Yahyaoui, Anish Acharya, Seema Amble, and Lauren Murrow

      Millennials and Gen Z have been hard-hit by the one-two punch of the 2008 and 2020 financial crises. That experience has radically shaped their approach to finances and their mindset around credit and debt. This episode explores how fintech founders are now designing products tailored to the financial challenges of younger consumers, from managing and avoiding student loans to building credit to saving and budgeting apps.

      Historically, students have largely been overlooked by traditional banks. Due to a combination of economic forces, predatory lending practices, and uninformed decisions, millennials have more outstanding student loans—and owe more money—than any prior generation. According to a poll released this week by the data intelligence company Morning Consult, just 46 percent of millennials believe their student debt was worth attending college.

      Amira Yahyaoui wants to change that. She’s the founder and CEO of Mos, a platform that allows students to apply for every government college financial aid program with a single application. In this episode, Amira joins host Lauren Murrow and a16z fintech partners Anish Acharya and Seema Amble to discuss how fintech can cut through bureaucracy, downsize student debt, and optimize—and ultimately automate—consumers’ financial futures from an early age.

      Show Notes

      • The student debt problem and how technology can help [1:29]
      • Why millennials and Gen Z have different expectations around finance [6:43], how traditional banks have responded [11:18], and how fintech apps are targeting students [13:10]
      • Apps that help students learn financial responsibility, as well as a discussion of alternatives to traditional four-year college degrees [17:56]
      • What fintech founders should consider when designing products for Gen Z [23:38]
      • What banking may look like in the future [25:15]

      Transcript

      The college debt problem

      Amira Yahyaoui: We decided to tackle the problem of paying for college and, more importantly, accessing higher education. So we think that money should not be the reason you decide to go or not. We want to make it free and we want to make it accessible. And we’re hacking the system to make it that way.

      Lauren Murrow: In that financial aid is such a maze and such an obvious pain point, why haven’t others come in and tried to compete on that front?

      Amira: Helping students go to college without debt is not a new idea. But honestly, the major reason is no one really wanted to build it. Most of those who build solutions either went to college with scholarships, or were able to pay for it, or didn’t go. And when you didn’t feel the pain, it’s hard to want to solve it.

      So a lot of noisy people in Silicon Valley, they will tell you, “Okay, the college degree is not necessary.” But then you look at their LinkedIn profile and they all went to Stanford. So a thing that you see, especially in Silicon Valley, is that those who are anti-college think if you can afford it, you go, but the rest of the population doesn’t need it. But that’s a big misunderstanding of why people go to college and why people need an education.

      The second thing is millennial fintech was a lot about lending. And that was, at the time, how companies like SoFi and others were created. Which is: loan providers are horrible, so we are going to be better loan providers. And millennials really loved it. But today, if you talk to Gen Z, it’s like, “Why should I pay that much? Why should I screw up my future with a $150,000 loan?”

      Seema Amble: I think what we also see is that students and parents alike don’t really know how complicated the process of finding funding is until they actually get to that point, navigating the FAFSA process, as well as the private loan process. People don’t know where to turn, what to apply for, even the difference between a federal loan and a private loan. And that process is unnecessarily complicated, if we’re trying to get people educated.

      Amira: Absolutely. The number one reason why people don’t access their rights in the world is bureaucracy. You have to spend your day on government websites with 1998 designs and no API and PDFs on horrible bureaucratic stuff.

      Lauren: We talked about how companies like SoFi targeted students with lending. But traditionally, why haven’t banks courted students in the past? Why haven’t they specifically designed tools for this demographic?

      Anish Acharya: I think that they did for a little while. You know, I signed up for a credit card in university and I got a T-shirt. And I think there was a lot of that credit card marketing happening in college campuses. And that, of course, was outlawed under Dodd Frank, because what they realized was, hey, your “free T-shirt” actually costs like $500 in credit card interest. And once that went away, banks really didn’t have a way to make money off of students. And because they’ve traditionally been so short-term oriented, they pulled out of the market altogether.

      Seema: And I think the other point, too, is they’re losing that touch point on the student side. So, before, the banks were getting subsidized by the government to provide student loans. And that got removed; in 2010, the federal family educational loan program ended. And so we’ve seen the government providing an increasing share of the student loan market.

      Lauren: How big is this market? Amira, do you have a sense of scale?

      Amira: So the U.S. government gives around $1 trillion dollars a year of aid. But in the student financial aid part alone, the total amount is $135 billion. And these $135 billion dollars are cut into very small checks—just an infinite number of applications.

      Anish: I feel like the magic of this country is that it’s so loosely coordinated that all of the best ideas just sort of, through market forces, percolate to the top. But anytime you have to do something that’s highly coordinated—like, let’s find one way to access all of these programs—it’s enormously challenging. And look, the same things happened with COVID, right? The countries that were able to create an enormously coordinated response have been really successful. And the countries that are intentionally very fragmented have been less successful.

      Amira: Absolutely. And if you think about financial aid, it’s the same thing. Why does a student need to spend every weekend filing forms and entering their name 15 times? Assuming that someone should deserve the right to have a higher education should not depend on how good you are at filling out stupid forms. And that’s the absurdity of it.

      Gen Z’s expectations around finance

      Lauren: I think part of it is changing of consumer expectations. This generation has embraced mobile banking and fintech in a way that previous generations did not. And they grew up in the wake of the last financial crisis, so perhaps they don’t have the same unwavering trust in institutions as their parents did. They’re often more leery of debt and credit, as we see younger consumers preferring debit to credit cards. And they grew up with the internet. So all this seems to set the stage for the rise of fintech serving this particular market.

      Amira: How do you expect the most educated, the most intelligent, but also the most cynical generation to just trust without facts? There is an incredible, beautiful change in consumer behavior, which is probably the biggest bullshit detector of all consumers. So if you are trying to trick the user in some flow or some ad, that just doesn’t work anymore.

      Seema: I think you’re touching on a really important point, which is trust and transparency, which we’re seeing not only in student lending startups, but in the new generation of consumer fintech startups overall. Millennials and Gen Z really need to believe that financial institutions are on their side and trying to help them navigate this, rather than all the gotchas and fees and fines to make money off of you.

      Anish: I think another interesting point is we live in a generally high-trust society, where people are willing to try new things. If you go to many other countries in the world—India is one that Seema and I have studied, for example—it’s not a high-trust society, which is why you see these massive conglomerates. They do like 25 different things: they make cars and sell cement and they offer financing. If you look at how trust is changing, though, trust is actually eroding in institutions and legacy providers of financial services. And trust in software and technology startups is really increasing. So I think it’s the best time to be building a company like this.

      Lauren: What are the specific consumer expectations that we’re seeing out of the younger generation? And how do you see fintech companies addressing that in product?

      Anish: I think that the old model of “hey, here’s a free t-shirt if you take my credit card,” you know, banks’ focus on owning a transaction is going out the window. I’m glad that Amira brought up SoFi because I think that while they did a lot of interesting things, they brought the old mentality of treating every opportunity to interact with students as a transaction. And, there’s only so far you can get in arbitraging the loan APRs that are being given to students that went to Harvard and Stanford. And I think that we’ve seen the results of that. [SoFi] built a big company but has struggled to build additional financial services because giving someone a better loan is great, but probably not enough to have a lifelong relationship with them.

      So I think that the expectation now is to have a deeper, more meaningful relationship. And look, I think consumers understand that companies have to make money, which is why models like subscriptions or even just charging consumers directly make a ton more sense. Consumers are actually smart, smarter than maybe we’ve given them credit for in the past. So I think there’s a higher expectation for the longevity of the relationship. There’s a higher expectation of value that’s delivered. But there’s also a higher willingness to pay, all of which is really good news for startups.

      Seema: I also think if you build trust early on with the student that can go a long way. The process of paying for college, navigating FAFSA and scholarships and grants was not always seen as a business. The whole student loan process was really seen as being more on the not-for-profit side, rather the profit side. And to Anish’s point earlier, personal loans and credit cards for students were really just thought of as something that a big top 10 bank would end up offering on campus or a local credit union would offer. And now fintech has the opportunity to weave something that was traditionally not-for-profit, and create a real product around it, and improve the students’ experience, not just from a financial perspective.

      And that really presents an interesting proposition in the sense that if you help students find money to pay for college, you can probably build a longer relationship with them over time and offer more financial services. But you’ve unlocked something that they would have an incredible amount of pain getting to themselves.

      Amira: And I think there is also a behavior that is very different, which is that usually companies try to win more than their users, right? And Mos is the opposite actually. The user wins more than the company And that is a different way of thinking. And I think that is also what is hard to understand for a bank that just wants to win more than you.

      I mean, I’ve been with bankers in the room when I explain what I do and I remember one banker who looked at me and said, “Economically, it doesn’t make sense.” And I’m like, it actually makes total sense! The bank of the future will be a bank that is able to transform $1,000 dollars in to $5,000 dollars, because a 1.2% gain is not interesting for this generation.

      Anish: It’s also interesting because banks’ whole business has always been two things: taking deposits and making loans. So even if you assume that they were able to do this it would just be very unusual for them to do it because it’s not their business.

      Seema: Because their mindset is: how do I create a transactional account or a financial product and sell it to you, rather than, how do we actually build a tool, or a product, or a service, that is actually solving the pain point first? Rather than being around the dollars and cents.

      Lauren: And now, of course, there’s a slew of fintech companies that are specifically targeting students. They’re tackling student debt, how to manage it, and more recently, how to avoid it. We’re seeing more around savings, particularly saving for tuition. There’s budgeting apps targeted to students, helping them establish credit early. And then of course, there’s lending apps with terms specifically tailored to students.

      How fintech is targeting students

      So one point I want go back to is the importance of building that customer relationship early. I think more fintech companies are recognizing that. And so we’re seeing companies like Greenlight or GoHenry or Step that are encouraging kids and high schoolers to start saving early. Can we talk a bit about that trend?

      Anish: I think that navigating the long-term relationship is a real thing. You don’t ever want students or individuals to feel like they’re in the kiddie pool of financial services. And when you graduate from college, a lot of things in your life change, you want to start being an adult. And I think you have to design the product, as well as the brand, to be one that actually can have that longevity. You don’t want it to feel like, hey, this is a constraint my parents put on me, versus, hey, this is something that’s actually going to enable me for the future.

      Amira: I actually think that if you are thinking about keeping a long-term relationship, starting too early is not the solution. I think you should start the first year of adulthood.

      Seema: And I think getting people early means just getting them at the critical juncture, which is, when people are applying for college they’re taking that critical step of managing their own financial independence. And so I think this is a really interesting time to build trust when they relied on probably their parents, and they’re taking control of that.

      Lauren: And the data backs that up, that a little less than half of college students report having any credit cards, and among those, about 60 percent got their first card when they were around 18 or younger, which suggests that many of them are starting their credit card experiences around the same time that they transition into higher education. But we’re also seeing increased parental dependence. As of July, 52 percent of 18 to 29-year-olds were living at home with their parents, according to the Pew Research Center. That’s a rate higher than the Great Depression. In addition, 6 in 10 adults are relying on financial help from their parents. How does that impact the theory that companies should be trying to reach out to these consumers and capture them at this transition moment when, in fact, many of them are perhaps not reaching that transition moment at the same point that past generations did?

      Seema: I think one thing to point out here is that, even if you applied for all the federal loans and scholarships available to you, there’s a good chance that you won’t get the full amount and there’ll still be a gap. And you’ll have to fund that gap, usually through a private loan. Usually you go through a bank, for example, to get it. But that requires a cosigner. Like, who’s the cosigner going to be on that loan? Probably your parents. And so you end up falling back on your parents if you can, in many situations. And the cost of living is going up. And so I think that’s part of the reason that you see them still reliant on their parents.

      Anish: In many other countries, kids depend on their parents for a longer period of time. So I think that if it leads to a better long-term financial outlook for the individuals, then it’s not necessarily a bad thing.

      Amira: What I think is going to happen is that they will be more responsible financially, just because financial decisions are not made alone. Actually the fact that a lot of those young adults now live with their parents I think will be very interesting in terms of how their financials will go in the future. I think Gen Z is going to surprise many, many, many people with how little credit they will be using in the future. I think they’ve seen their parents crushed by debt. They are super aware about the consumerism and all of that.

      Anish: While I think that you’re right, there may be less of a functional need to get credit, I do think that it’s an important sort of emotional and psychological milestone to start to establish that credit. So look, while I think they may take less credit, I still think that they’ll probably participate in the system by establishing credit, at least in a lightweight way.

      Seema: And I think establishing credit isn’t just about: okay, now I have a credit score that’s in the prime segment, and now I can take on more debt. It’s also preserving optionality, which I think isn’t something that you totally understand when you’re 18. So, you know, you need to build a credit score if you one day want to buy a home—and maybe Gen Z doesn’t—but you have that optionality to do so. Or for an auto loan or any of these big purchases that a lot of times when you’re 18, you’re not really thinking about.

      And we’re seeing, on the product side, a number of products that that are helping students think about how to build credit. Because I think that’s not something that students think about. Things like: the account I’ve had opened the longest is actually what’s driving my credit score. And how do you build a good credit score? And so these are all things that you have to learn, which is an opportunity for fintech.

      New paradigms and alternatives

      Lauren: I like that you brought up that idea of financial responsibility. There are many fintech companies that are designing budgeting apps specifically with students in mind, which are offering and things like ways to track their spending or setting goals and challenges, and sometimes peer comparisons on spending habits that are anonymized. So it’s interesting to see for those that perhaps don’t have that parental backstop, there are also companies that are seeking to tackle this idea of financial responsibility, and saving, and credit, in a way that appeals to a younger demographic.

      Anish: It’s funny, we’ve talked a lot about this. I think to some extent, yes, budgeting is important, but it feels like the much bigger lever is ensuring that students are getting access to every single dollar of financial aid that’s available to them. I think we should actually focus on bigger ways and bigger moves to actually assist these people.

      Lauren: If the existing budgeting apps are insufficient and you want to think bigger, where are those areas that you see opportunity?

      Anish: I’ll give you one point that’s really piqued my interest. Just as 20 years ago, blogger and other technologies made every individual into a publisher, perhaps what’s happening today is that every individual’s becoming an investor. And look, I think that the next generation is much more savvy and is going to participate in investing at a higher rate. And I think instead of talking about controlling costs, let’s talk about increasing “revenue.” And there are a lot of ways to do that that I think are interesting.

      Seema: Yeah, I think there’s still a lot more to be done on the employment side.

      Lauren: That’s a great point, Seema, in that rising college seniors are now facing the worst job market in modern history.

      Seema: Traditionally, that was like, oh, I’m going to go work at a library or the coffee shop. Now, we’ve seen an explosion of online tutoring, for example, in COVID. I think we’ve seen a lot of platforms pop up, but I don’t know if they’ve necessarily connected with students specifically. To Anish’s point, it’s a lot around: how do you increase the amount of revenue for the student, in addition to the loans, so that they can become more entrepreneurial?

      I think you’re also seeing a change in how education is being offered, especially in COVID, where kids aren’t able to actually go to school in many cases. But it’ll be interesting to see what traditional education looks like, and also the necessity to pay for traditional education versus am I going to go through a coding boot camp or what we would traditionally call vocational training, but really skills based. And I think alongside of that, we’re also seeing things like ISAs pop up. An ISA is an income sharing agreement. Essentially, the student borrows money from the institution to fund their education. In exchange, they pay a percentage of their salary over time after graduation. And it’s up to a certain cap, so if you end up making a lot of money it doesn’t scale with it.

      Anish: I think the important point here, though, is it’s a false dichotomy to say, hey, either you deserve to go to college or you don’t and you should go to coding boot camp. I think consumer choice has to be the number one consideration. And I do think there’ll be a barbelling that happens, which is there’ll be a bunch of students that want the brand or the experience of going to a Stanford, Harvard, and they’ll do that. There’ll be a bunch of students who want either the life experience of going to a small liberal arts college or perhaps the sort of vocational training of going to something that’s much more coding boot camp oriented. And I think the schools in the middle are going to be the ones that struggle, because consumer preference is going to clarify around choosing those options.

      Amira: I agree. I think we are in the Stone Age of disrupting colleges. No student who has a choice between Stanford and another solution will pick the other solution. We’re very, very, very far from that.

      Lauren: I think a big part of that we have to acknowledge is, particularly this year, the widening gap between income levels and tuition costs, as many tuition rates remain the same. It’s not just a dichotomy of “I want the experience of college” or “I don’t want to be on Zoom.” I think there are also very real financial considerations that these companies are taking to task.

      Student debt in the U.S. now totals more than $1.6 trillion. And 7 in 10 college seniors are graduating with debt. So I do think that Gen Z, in particular, views their parents and, to some extent, millennials as a cautionary tale.

      Amira: Also, in the future I think they will be pretty critical of companies that are just selling debt all the time to them in a pushy way, and making it super easy, and making it super accessible.

      Anish: Yeah, I think there’s also an interesting example of a principal agent problem here, which is the government has income-based repayment plans for student loans, but the channel for activating them is through servicers of student loans. And, you know, servicers of student loans don’t really have an incentive to move people on to plans like this. So even when the government has the best of intentions, their channel to their “customer,” their citizen, is very inefficiently delivering the information.

      Amira: Yeah, absolutely. I mean, getting student loans through the government is the least bad solution, right? But the solution is to get an education without paying out of pocket and, most importantly, without taking loans. Before considering any loan, even the best loan possible, they need to look at all the other solutions they can use to pay for college. And there is a ton of free money out there.

      Advice for fintech founders

      Lauren: So if we’re building more tools for this increasingly savvy, increasingly digital generation of consumers, what should founders consider when they’re designing financial products for them?

      Anish: A transparent business model. Being clear about how you’re making money and making sure that your incentives are aligned with your customer in the short and long term.

      Seema: I think one company that’s done a good job of connecting with younger generation has been Cash App, not only on transparency around the fees, but on the marketing side. Both in terms of product—so they offer things like fractional stocks, new products that are on the market and not available necessarily other ways—but then they do the $Cashtag and the Cash drop on Twitter. And it’s an inherently social experience in a way that other financial products just haven’t been able to reach young consumers.

      Amira: I would also add it has to really work, not marginally work. If you are building a product that is 5 percent better than what others have built, you’ll get nobody. There are so many copies of every single app out there, especially in fintech. Just the number of new banks—they all look the same. You will not make somebody switch for 10 percent better. With this generation, you really need to build something that is worth the attention span and as I said, the bullshit detector. We only sell to students and we need to build the product that they want to buy.

      The future of banking

      Lauren: As more fintech startups design with this mobile-first consumer with high expectations in mind, do you think this will have a long-term impact on traditional financial services? We’re seeing a bit of that already.

      Anish: Not really. Of course we should have more digitization. Yes, of course we should have more innovation. The challenge for banks is not that they lack talent, it’s the things that would be most “innovative” or create the most utility for their customers are things that are unfortunately bad for their short term revenue. So I think there’s a huge structural disincentive for them to compete directly. And I think what happens as a result is that banks end up being further commoditized, where they are just loan providers who are indistinguishable, and companies like Mos end up being the intermediary and owning all the economics of helping students navigate refi, etc.

      Seema: And, and owning the customer relationship.

      Anish: 100 percent.

      Lauren: So all of this indicates this revived urgency, if we haven’t already seen it, for companies that help the younger generation manage their finances, save earlier, avoid and pay down debt, build credit, afford real estate, all this. When we’re talking about designing for Gen Z, or for students, where do you still see opportunity? What’s next?

      Amira: I believe that the banks of the future will not look at all like the backs of today. Having your money on even a very well-designed app and basically earning some perks here and there, it’s not worth it.

      Lauren: What is your vision for that bank of the future? What will it look like?

      Amira: The bank of the future will not only manage your money or help you save, but it makes your money multiply. You come in, you’re a student, and your bank just applies you to everything to fill out your account so that you can pay for college. So I really think that those services should be part of whatever is like your money management solution.

      Seema: And I think that involves also bringing together a lot of your accounts. We’re investors in Tally, which helps you manage your credit card debt. But being able to manage all your debt across sources, or even your income streams. Wealthfront just launched a product, as well, on the investment side. But this concept of automation of your personal finances is very much still in its nascency.

      Amira: Another one is about accessing the job market. Today, colleges help you for your first job. And in the future every person will have 20 jobs during their lifespan. How you get trained for all of them is going to be something interesting.

      Seema: And I think a lot of that starts when, people are students or even probably even younger, in high school. But this idea of not just taking one job, but you’re glomming together a variety of, you know, a tutoring job, a passion economy project, from an early age.

      Anish: Also, just no longer having just one job for life. That’s over, you know? And people are going have multiple jobs in their career and maybe multiple jobs at the same time, and they won’t necessarily have to congregate in the same city centers. The way that we work and the way that we make money is going to change dramatically.

      Seema: I spoke on the employment side, but then also on the education side. Even if four-year education may continue in certain ways, I think people will add on their own forms of education. And so looking for ways to pay for education is going to have to adapt. It’ll generally just be more modular.

      Anish: I agree. The other thing that I think is interesting is, how has leverage changed over human history? The most historical model of exerting leverage was through labor—having a lot of people working for you. The second was through capital, which is “the rich get richer.” And I think the new model for leverage is software. You know, perhaps in the last generation, you could only own an investment property if you had achieved a certain degree of wealth. But now with the fractionalization of things like investing in real estate, almost anyone can participate in the economics of an investment property. So I’m really bullish on software making things that were previously only available to people that had a lot of capital available to all people. And that’s why I think the mentality of hustle culture and “everyone’s a founder” and all of that is a hugely positive thing and very trend-aligned with what’s happening in technology.

      Lauren: Thank you all for joining us on The a16z Podcast.

      Anish: Thank you, Amira. Thank you, Seema.

      Seema: It was great to join you guys.

      Amira: Thank you so much.

      • Amira Yahyaoui

      • Anish Acharya is a general partner at a16z. Prior to joining the firm, he served as a GM at Credit Karma. He also founded SocialDeck (acquired by Google) and Snowball (acquired by Credit Karma).

      • Seema Amble is a deal partner at a16z where she focuses on fintech companies. Prior to joining the firm, she worked at Goldman Sachs and LeapFrog Investments.

      • Lauren Murrow is an editor at Future. She oversees posts, podcasts, & special projects for a16z's consumer and fintech teams. Previously, she was a senior editor at WIRED, where she edited op-eds and features.

      Degrading Drugs for Problematic Proteins

      Carolyn Bertozzi and Lauren Richardson

      In Bio Eats World’s Journal Club episodes, we discuss groundbreaking research articles, why they matter, what new opportunities they present, and how to take these findings from paper to practice. In this episode, Stanford Professor Carolyn Bertozzi and host Lauren Richardson discuss the article “Lysosome-targeting chimaeras for degradation of extracellular proteins” by Steven M. Banik, Kayvon Pedram, Simon Wisnovsky, Green Ahn, Nicholas M. Riley & Carolyn R. Bertozzi, published in Nature 584, 291–297 (2020).

      Many diseases are caused by proteins that have gone haywire in some fashion. There could be too much of the protein, it could be mutated, or it could be present in the wrong place or time. So how do you get rid of these problematic proteins? Dr. Bertozzi and  her lab developed a class of drugs — or modality — that in essence, tosses the disease-related proteins into the cellular trash can. While there are other drugs that work through targeted protein degradation, the drugs created by the Bertozzi team (called LYTACs) are able to attack a set of critical proteins, some of which have never been touched by any kind of drug before. Our conversation covers how they engineered these new drugs, their benefits, and how they can be further optimized and specialized in the future.

      Show Notes

      • How conventional drugs work, and how PROTAC targets proteins differently [1:46]
      • Discussion of how PROTAC can only reach proteins within the cell [6:00], how LYTAC targets external proteins [9:56], and how LYTAC pinpoints specific proteins for degradation [12:54]
      • Implications for future use of LYTAC [16:37], and what work is needed to turn it into a therapeutic treatment [20:29]

      Transcript

      Hanne: Hi, I’m Hanne, and welcome to Bio Eats World.

      Lauren: I’m Lauren, and this is the first episode of “Journal Club.”

      Hanne: So, tell me. What’s “Journal Club” all about?

      Lauren: So, “Journal Club” is on Thursdays, and this is where we take a recent scientific article and discuss it, either with the authors of the paper, or with our own internal experts here at a16z. And we highlight what the paper shows, what new opportunities it presents, and how to take those research findings from paper to practice.

      Hanne: Okay. So, tell me — what’s this first paper all about?

      Lauren: The first paper is titled “Lysosome-targeting chimaeras for degradation of extracellular proteins,” and it was published in Nature.

      Hanne: That’s a lot of words. What do they actually mean?

      Lauren: The basic idea is that diseases are often caused by proteins that have gone haywire in some way. So, there’s either too much of them or they’re present in the wrong place or at the wrong time. And the idea here is to create a new kind of drug to degrade those proteins. So, if there’s too much of the protein, you’re reducing the levels. If it’s in the wrong place or the wrong time, you’re removing it from that area. And that’s a really exciting new type of drug molecule.

      Hanne: Okay, cool. So, what’s important about this paper? How is this moving us forward?

      Lauren: This paper is really exciting because it’s targeting a whole new class of proteins, some of which have never been able to be touched by any other kind of drug.

      Hanne: And who is the guest joining you for today?

      Lauren: Right. Today, we have the senior author on the paper, Carolyn Bertozzi, who is an amazing scientist. She’s a professor at Stanford and her work focuses on creating new methods to perform controlled chemical reactions within living systems. So, we’re going to lead off with Carolyn describing how conventional drugs work.

      PROTAC vs. conventional approaches

      Carolyn: So, most conventional medicines act by binding to a target — a pathogenic driver. That’s a protein in your body that’s contributing to a disease. And they act by what’s called occupancy-driven pharmacology. They bind to that target and block its function. So, ibuprofen binds to an enzyme and blocks its activity, which then blocks an inflammatory pathway.

      Lauren: So, the normal typical drugs are working by binding proteins and blocking their activities. But in the last 10 years, we’ve seen some really exciting alternatives to drugs that rely on this specific model, with the most well-known and the most well-developed being what’s called a PROTAC, or a proteolysis-targeting chimaera. So, how do these new drugs differ from what we just described — the standard typical drugs?

      Carolyn: So, that concept came out of academic labs in the early 2000s, and the two people who published the defining papers in this area were Craig Crews from Yale and Ray Deshaies, who at the time was at Caltech — now he leads research at Amgen. And they had this idea that another way to shut down a pathogenic protein would be to target it for degradation. And around that time, there had been some breakthroughs in our understanding of how nature normally degrades proteins, because she has to be able to do that — new proteins get made, old ones get degraded. And a central mechanism for degradation of proteins inside the cell is that they get marked with ubiquitin chains, and that’s a signal for the proteasome — which is like the meat grinder inside of the cell — to chop up these proteins and destroy them. And there are enzymes that put these ubiquitin chains onto proteins that are destined for degradation.

      And so, what Crews and Deshaies realized is that you could build a molecule that artificially bridges the gap between a target protein and this ubiquitin machinery. And with that molecule, you could basically get a protein ubiquitinated intentionally, and therefore degraded. So, that was their conceptual idea.

      Lauren: So, in the course of the normal function of the cell, you have proteins being produced, but you also have proteins being degraded. And so, one of the main mechanisms for degrading protein is by the ubiquitin-proteasome system. And that’s where the cell says, “Degrade this protein by adding ubiquitin molecules onto it,” and that pulls it to the proteasome where it gets chopped up. But what a PROTAC does is, it’s a molecule that can bind to target protein — so the one that you want to degrade — and brings the enzyme to it that adds the ubiquitin tag — adds the flag — and then that brings it to the proteasome to be degraded.

      Carolyn: Right. And the reason that was so transformative is that not all proteins are easy to block, actually, with drugs. There are lots of proteins that are not enzymes, and they don’t even have a pocket, really, where you could put a drug and it would block the function. So, the cool thing about these PROTACs is that they don’t have to bind in a place that would block its activity, but instead bridges the gap to an enzyme that puts the ubiquitin on and drives the degradation. So, the promise, really, is that the PROTAC — or the targeted degradation approach — expands the druggable proteome because now more proteins can be drugged, because you have this other way of doing it — through degradation and not just blocking.

      Lauren: So, you’re using the endogenous mechanism that the cell already has for flagging proteins that you want to be degraded, and using it now to target a much wider range of proteins than you could if you were only able to target those that have, you know, a really nice pocket that could be targeted with an activity inhibitor.

      Carolyn: Right.

      The benefits of LYTAC

      Lauren: So, what are some of the limitations of these approaches?

      Carolyn: Well, the targeted degradation field began with the PROTACs, but it has expanded over the last 20 years to include other types of protein degraders — but all of these processes function on proteins that are inside the cell — in the cytosol or in the nucleus. And meanwhile, there’s this whole other world of proteins which are outside the cell. So, these are proteins that are displayed on the cell surface — the membrane-associated proteins — many of which, the majority of the molecule is outside, presented on the surface, where it’s not accessible to the proteasome. And as well, there are many proteins that are just completely secreted by the cell and just released into the extracellular space, and those extracellular proteins are about 40% of the human proteome. So, that’s a pretty big chunk of the pie that is not available to the PROTAC strategy.

      And many of these proteins — these extracellular and cell surface proteins — are important targets for drug development. And, you know, my lab had been working on a variety of different cell surface molecules and secreted molecules that contribute to things like cancer immune evasion, for example. And many of the molecules we wanted to drug were really not druggable using the conventional blockers. And that’s where the lysosome-targeting chimaera, LYTAC, research started.

      Lauren: I see. So, the PROTACs that you described are a really exciting new modality, but they are limited in that they can only target the proteins that are within the cell. And there’s this huge world of proteins that just are not available to be targeted in that way. And they aren’t ones that rely on occupancy of, like, a particular binding site — they can’t be targeted by those types of drugs either. So, they’re really kind of an unmet need for drugs to target them.

      Carolyn: I would go even further and say sometimes even targets that can be drugged with a blocker, you can get a more potent effect with a degrader at lower doses, right? So, even secreted and cell surface molecules that have been successfully drugged with monoclonal antibodies, you might actually do better if you convert over to a degradation strategy.

      Lauren: Why do you think that is? Why do you think they’re better than activity modulators, or is that not known?

      Carolyn: Well, I think with occupancy-driven pharmacology, you can’t ever get, like, 100% of the target protein blocked. There’s always an equilibrium, and you have to constantly pump the system with enough drug to keep the occupancy as saturated as possible. By contrast, the degrader can bind to a target and get rid of it, and then bind to another target, and get rid of it, and bind to another one, and get rid of it. So, you’re just reducing the level of the target protein, but because there’s the potential for one drug molecule to mediate the degradation of multiple targets, you could get a deeper inhibitory effect in principle. And that has now been borne out, even in some early-stage human clinical studies, with PROTACs. The same could very well be true with LYTACs. Of course, it’s a much earlier technology, so we don’t know that definitively, but there’s — I think — a rationale for thinking that way.

      Lauren: Yeah. That’s kind of like pharmacodynamics 101, that you have a reversible inhibitor, and you’re going to have this equilibrium, but these degrading molecules, you know — they don’t get degraded when they tag the protein for degradation. They have a benefit of — one degrader molecule can target a huge number of target molecules. So, that’s really interesting, that even in, like, a head-to-head comparison on a known druggable target, that you can possibly get a better effect by degrading as opposed to inhibiting.

      How LYTAC targets proteins

      So, now that we have the background on why we need this new type of drug, why you decided to go after extracellular and membrane-associated proteins — let’s get into the details of how you develop these molecules. And, as we mentioned, the PROTACs co-opt this endogenous pathway, the ubiquitin-proteasome pathway — but they can’t reach these proteins outside the cell. So, what cellular pathway did you co-opt to degrade those proteins?

      Carolyn: So, again, nature degrades these extracellular and circulating molecules, and she does this through what’s called the endosome-lysosome pathway. So, cells will basically internalize and engulf molecules from the extracellular space into endosomal vesicles that go through a maturation process to become the lysosomes. And the lysosomes — people from their cell biology classes might recall — that’s the organelle within the cell that has a lot of degradative enzymes. So, lysosomes can degrade proteins, polysaccharides, lipids. There’s a lot of hydrolases within the lysosome.

      And so, we conceived of an idea where we would develop bifunctional molecules, where one part binds the protein that you want to degrade, and the other part binds a lysosomal trafficking receptor system. So, that’s the key — is that lysosomal trafficking system. And it turns out that in human biology, there are about a dozen known receptors whose job it is to grab stuff — either from the membrane or from the extracellular space — and pull it into this endosome-lysosome pathway for degradation. And so, what we have done is hijacked those pathways by basically building molecules that interact with those receptors, and then attaching them to a molecule that binds a target of interest. So, that’s the structure of the LYTAC — a binder on one side for the target, a binder on the other side for a lysosomal trafficking receptor.

      Lauren: Right. So, nature has already come up with a way to degrade the proteins that are membrane-associated and extracellular, and you just developed a mechanism that allowed you to say which protein you want to degrade and then extracting it from the extracellular space and degrading it inside the cell.

      Carolyn: Yeah. And one of the best known lysosomal trafficking receptors is the so-called mannose 6-phosphate receptor. And mannose 6-phosphate is a sugar epitope that is found on lysosomal enzymes, and that allows them to be trafficked to the lysosome by this receptor — the mannose 6-phosphate receptor.

      Lauren: So, you have this sugar molecule that if you attach it to a protein, that’s going to take it into the lysosome. So, how did you engineer the specificity to target the protein that you wanted to the lysosome?

      Carolyn: So, you need a binding molecule that is very specific, and ideally also very high affinity, against your target of interest. And, you know, in our early proof of concept studies, we chose targets to degrade for which there already were high affinity, high specificity antibodies available — several of which are already approved human medicines. So, for example, we’re interested in the epidermal growth factor receptor as a target for degradation. This is an important cancer target. EGFR — it’s overexpressed or mutated in many cancer types, where it’s driving the proliferation of cells. And we made a LYTAC out of a human drug called cetuximab — it’s an antibody against EGFR that is used, you know, in the oncology setting. So, that process of taking an antibody against a target and just decorating the antibody with the mannose 6-phosphate groups — that converts it to a LYTAC.

      Lauren: Great. So, antibodies are molecules that our immune system produces, and they are incredibly well-tuned to bind one specific protein — and there are many drugs that are actually antibodies — but their main function is to just block that protein. And what you did was you took that therapeutically active antibody and added the glycan molecules that you needed to turn it into a LYTAC. So, now not only is it blocking the protein, but it’s shuttling it into the lysosome to be degraded. It almost gives it, like, an extra function— like, making it even more effective at disrupting their target’s function.

      Carolyn: That’s right. Also, the more we learn about biology, the more we are appreciating its complexity. And I think we also are now understanding that most proteins have functions that are not just binary, you know — like an enzyme is either on or off. Most proteins have multiple dimensions to their function. They interact with other proteins. So, when you block a protein through an antibody, or through a small molecule inhibitor, there are probably other interactions of that protein that you’re not affecting, which still contribute to the biology. And when you degrade the protein entirely, you take away all those dimensions of its function. And so, it’s not just that a degrader can be more potent than the inhibitor in an axis of biology — I think the degrader can have more axes of an effect.

      Lauren: Do you think that that could lead to possibly off-target effects of disrupting, kind of, a bigger network than you anticipated?

      Carolyn: That’s a good question, and I guess it depends on where you draw the line between on-target and off-target. Because, take a protein like EGFR. The biology of that receptor is driven by its interactions with other components of the signaling pathway. You know, EGFR binds its ligand — the epidermal growth factor — and the consequence of that is, that triggers a signaling cascade. And so, if you inhibit the activity of EGFR by just blocking, you don’t affect any of the downstream signaling biochemistry. 

      However, if you drive the degradation through the LYTAC approach, and some components of that signaling machinery come down with it, that is actually a direct hit — I would say that’s on target, right? Because you’re hitting not just EGFR, you’re hitting the complex that drives its biology, right? So, again, the biology is never transacted by a protein in isolation — it’s by that protein and the network of its interactors. So, I would argue that if you can degrade some of its interactors, it’s a more profound influence that’s on-target.

      Possible future applications

      Lauren: So, now that we’ve talked about, you know, the details of your study, how you develop these bifunctional ligands that can bind to a specific target protein and shuttle it into the lysosome for degradation — let’s zoom out and put this research into the broader perspective. What are some of the new opportunities that this work provides?

      Carolyn: We’re now exploring therapeutic applications of the LYTAC technology, and we’re interested in extracellular targets that have been very — either difficult, or really just impossible to drug, and there really is no option right now for patients for certain disease settings. So, for example, we’re very interested in diseases that involve aggregation of proteins in the extracellular environment. Proteins that in their misfolded or unfolded forms lead to toxic aggregates that can cause tissue damage. And so, these are diseases that are often called amyloid diseases. The ones that are most familiar to people would be neurodegenerative conditions like Alzheimer’s disease, Parkinson’s disease — it’s been very difficult to figure out, you know, how do you get rid of these protein aggregates that are pathogenic in the extracellular space? They’re not really amenable to inhibition — the process by which they form is often not well-understood. You really just want to get rid of them, right? You want to degrade them. And I think the LYTAC approach is perfectly situated to take on peripheral amyloid diseases. 

      For example, there’s a condition called light chain amyloidosis. Antibodies have a heavy chain and a light chain. So, in patients with this condition, there’s too much light chain all by itself, and it’s not stable, and it’s forming amyloid aggregates — which deposit in organs throughout the body and they’re toxic. The standard of treatment for these patients is very poor. So, we think the LYTAC approach could be interesting in that setting.

      Lauren: That’s a perfect example because those light chains don’t have an enzymatic function. They don’t have a nice pocket that you would be able to stick a drug in. So, the ability to pull those out of the extracellular space and degrade them with a LYTAC sounds like a perfect match between disease physiology and drug modality.

      Carolyn: Yeah. So, that’s an example of a, sort of, secreted pathogenic molecule or system of molecules. There are other membrane-associated targets that we think the LYTAC is well-suited toward. And one class of molecules that my lab is really interested in are called mucins. These are transmembrane glycoproteins that are huge, and they’re kind of the giant redwood trees of the cell surface, so to speak. And they’re known to be associated with cancers. And cancers that overexpress these mucin molecules — they tend to be very aggressive and very difficult to treat. And we’ve done a lot of work to understand, like, what’s the function of these mucins that’s oncogenic. And the bad news, from the perspective of drug discovery, is that a lot of the biology of these mucins is a physical biology. So, they’re pathogenic because of their stiffness, and their rigidity, and their physical effects on the cell surface — not because they interact with a receptor, for example, which maybe you could block, right? 

      And so, what do you do when the function of the molecule is a physical one, and not a biochemical one? And I think this is where you just want to get rid of them. I think you just want to degrade them. And fibrosis, right — that’s a disease setting where there’s pathogenic accumulation of collagen scarring. And you know, that’s hard to think about — how to drug that, you know, at least at the end point of the disease, where you have this material that you really just want to degrade. And so, again, I think a LYTAC strategy would be interesting to test in that setting.

      Lauren: A lot of really important applications for this. So, what are some of the elements of the LYTAC design that still need to be optimized to turn them into therapeutics?

      Carolyn: So, this was the “version 1.0” of the LYTAC technology, and the work that’s now going on is basically the second- and third-generation improvements. And those improvements have taken several forms. So, first of all, we’re interested in improving the structures. So, the second-generation LYTACs have a new chemistry, so that the conjugations are site-specific — that we can engineer the part of the antibody that actually gets coupled to the mannose 6-phosphate groups. And with our new chemistries, we can make different geometries of LYTACs and find what is the best geometry for a given target — and it probably will be target-dependent.

      So, we’re kind of now writing the rulebooks, and in the publication, the LYTACs we made are built from these known antibodies. We are now developing LYTACs from other kinds of binders — including small molecules that might otherwise have been blockers — we’re now converting them to degraders through the LYTAC approach. Another dimension that we’re expanding upon is the lysosomal trafficking receptor that we hijack. So, the mannose 6-phosphate receptor was a great starting point — it’s expressed in virtually all cell types. But there are other systems that are more specific for different cell types or different tissues. So, our next LYTAC family are targeting a receptor called the asialoglycoprotein receptor, which is a liver-specific lysosomal trafficking receptor. And we have a preprint that we posted on ChemRxiv on this new generation of LYTACs.

      Lauren: Yeah. Liver-specific makes a lot of sense because we were talking about fibrosis — liver fibrosis is a huge problem, and that’s caused by too much collagen in that area that you want to break down. But you don’t want to break down collagen everywhere in the body, you know — that’s really a critical molecule. You could get wrinkles, God forbid! <laughter> It’s really important in your skin, it’s really important in your joints. So to have that specificity of where you want to target the degradation is really important, and an additional, like, strength to this approach.

      Carolyn: Yeah. And I think that then hints to a broader universe of LYTACs that target different receptors that are tissue-specific in different settings. That’s the tip of, hopefully, a big iceberg of interesting new degraders.

      Lauren: And more to come. So, we’ll end with — what is the key take-home message from this article and from our discussion today?

      Carolyn: I think the most important point is that this exciting, still relatively young field of targeted protein degradation has just been set free from the confines of the cell. So, extracellular proteins should now be added to the list of potential targets for a degradation strategy, and we hope with the LYTAC technology that we can bring added benefit to patients.

      Lauren: Well, thank you so much for joining me today on “Journal Club.” I really enjoyed our discussion, and I’m so excited to see what comes out of this research.

      Carolyn: Thank you.

      Lauren: And that’s a wrap for the first episode of “Journal Club.” If you enjoyed this episode, please subscribe, rate, and review wherever you listen to podcasts. And to learn more about how biology is technology, subscribe to our newsletter at a16z.com/newsletters.

      • Carolyn Bertozzi

      • Lauren Richardson

      The Biology of Aging

      Laura Deming, Kristen Fortney, Vijay Pande, and Hanne Winarsky

      Welcome to the first episode of Bio Eats World, a podcast all about how biology is technology. Bio is breaking out of the lab and clinic and into our daily lives—on the verge of revolutionizing our world in ways we are only just beginning to imagine.

      In this episode, we talk all about the science of aging. Once a fringe field, aging research is now entering a new phase with the first clinical trials of aging-related drugs. As the entire field shifts into this moment of translation, what have we learned? What are the basic approaches to developing aging-related drugs? How is studying aging helping us understand diseases like cancer and Alzheimer’s — and increasing the amount of time we are healthy — today?

      In this conversation, Laura Deming, founder of The Longevity Fund; Kristen Fortney, co-founder of BioAge, a clinical-stage company focused on finding drugs to extend healthspan; Vijay Pande, general partner at a16z; and host Hanne Winarsky discuss the entire arc of aging science from one genetic tweak in a tiny worm to changing a whole paradigm of healthcare delivery.

      Show Notes

      • Overview of the research on aging [1:54] and the current state of the science [4:16]
      • The three most common research approaches [6:19], why this field is expanding so rapidly [8:21], and possible applications for disease treatments [11:00]
      • Discussion of pure aging research vs. treatments for disease [14:46]
      • Getting this science into the healthcare system [18:42] and issues with research funding [22:16]
      • The types of entrepreneurs needed to expand the field [25:23]

      Transcript

      Hi, I’m Lauren.

      Hanne: And I’m Hanne, and this is our first episode in the new podcast “Bio Eats World,” where we talk all about how biology is breaking out of the lab and clinic and into our daily lives — and really on the verge of revolutionizing our entire world in ways we’re only just beginning to imagine.

      Lauren: So, Hanne, the title of this first episode is “The Biology of Aging.” What aspects of aging are we gonna be discussing today?

      Hanne: Well, really we’ve been trying to dream up ways of slowing down aging for as long as we’ve been aging, right? But the field of studying aging as a science is pretty new. So in this episode, we look at the entire, kind of, biology of aging — what we’ve learned; what’s reality; and what is translating into actually increasing our health span, and potentially — one day — possibly [slowing] down aging.

      Lauren: What’s healthspan, and how’s that different from lifespan?

      Hanne: Your first thought when you think about studying aging might be how we might slow it down, but really the way a lot of people in the field think about it is increasing our healthspan — which is the amount of time that we live healthy. What’s really interesting about this episode is, it’s about not just increasing healthspan and age span, but what we’re learning about disease — and particularly chronic age-related diseases — that might help us be healthier today.

      Joining me for this conversation is Laura Deming, founder and partner of The Longevity Fund; Kristen Fortney, founder of BioAge, a clinical-stage company focused on finding drugs that extend healthspan using machine learning; and Vijay Pande, a16z general partner on the bio fund.

      Lauren: Were there any insights from this episode that changed the way you think about aging yourself?

      Hanne: Yeah, well, I definitely enjoyed hearing about the drug already widely available that really might increase our lifespan. And I also loved hearing about what the difference between Benadryl and Unisom is. So we start with a little bit of a history of the field, talk about where it’s come [from], and where we are today.

      History of research on aging

      So, where actually are we in the biology of aging today? There’s been a big surge of talk, even over the past few years, about what the science of longevity is — how it’s developed — but where are we actually today?

      Vijay: Mortality is, like, this thing that philosophers opined [about] for millennia, but yet the biology of aging seems new. <laughter>

      Kristen: Right. New, insofar as it’s new that anything actually works, I guess, right? One of the earliest discoveries in aging research that goes back decades is that if you could severely restrict food intake in animals — calorie restriction — they would live substantially longer. But it’s only been fairly recently that we’ve been able to actually intervene, and actually impact how long a mammal can live. And one of the interventions that was first shown to work in mammals is parabiosis — exposing old mice to young blood. And that really was first discovered 50 years ago.

      The major acceleration came during the 1990s, the 2000s — and it’s mostly attributable to the first finding, you know — Cynthia Kenyon, Gary Ruvkun, Tom Hughes — that you could delete a single gene in the worm C. elegans and double its lifespan. Everyone thought aging — so complicated. You know, how are we going to have a dramatic impact on aging when it’s really all of these different systems and processes that are going wrong simultaneously?

      And then, you know, wow — wait a minute. This one tweak, and then suddenly this massive difference in lifespan. So a lot of invertebrate geneticists went into the fields and mapped out all these longevity genes that impact worms, flies, and yeast, which is awesome. But now, you know, which of those translate to humans? Those are the ones that matter for translation.

      Laura: Going back to, kind of, the history of the field — you, kind of, have these really — you know, sort of, highly-advanced intellectuals, going to the field and then, kind of, losing a lot of their momentum forward, practically — Nobel laureates like Elie Metchnikoff, claiming that gut bacteria, kind of, control aging. And maybe that’s coming back around now in some areas of current biology, but back then, it’s not as well supported. It’s only recently that you started to have the traction in the field to make specific discoveries.

      That period of time was just so critical to the field’s birth. Cynthia Kenyon, when she was making these first studies, was told, “You’ll fall off the face of the earth, literally, if you pursue this research and you do the study.” And if you look at her first paper, she was the lead author because no grad student was willing in her lab to do the work. That was such a controversial first step to take as a, you know, young principal investigator. That was how unexpected it was <Mmhmm.> that people really thought that it would be the end of your career to, kind of, go into this field — and then she, kind of, you know, started it anew.

      Hanne: They didn’t even want to touch it.

      Kristen: Yeah, exactly. Worse than unexpected — like, bad science.

      Current state of the science

      Hanne: So, can we talk about what that traction actually is looking like right now? What is the most promising traction?

      Laura: I think one thing that we feel really strongly is — this is the critical decade. Patients are for the first time receiving drugs that were developed in the context of aging biology. And it’s fascinating to watch these first clinical trials occur, where companies are actually developing drugs.

      And when that first patient gets an actual clinical benefit, we’re gonna see — you know, people actually affected by these, kind of, ideas that have percolated in the field for decades.

      One of the, kind of, examples of this that’s most, sort of, prominent in the field was a trial testing a drug called Metformin in the elderly. And so it’s actually looking at all-cause mortality, not just a specific disease as an endpoint. Metformin itself, you know, is this drug which retrospectively has been shown to be somewhat correlated to a decreased mortality in, for example, diabetic patients.

      Kristen: Well, it was discovered just by analyzing health records, right? So, so…

      Vijay: Which itself is kind of fun.

      Kristen: Which itself is, like, yeah — that’s a great way to find, sort of, repurposed drugs.

      Vijay: Yeah, and who’s living longer.

      Kristen: Exactly. Yeah. So it’s one of these drugs that millions of people have been taking for decades. You can actually go back in time and ask the question — you know, are people who are on Metformin living longer? And they are, and it’s kind of amazing. So that’s, sort of, where the whole hypothesis for this compound came from that’s now being tested in the clinic, which is so exciting.

      Hanne: I gotta go get me a prescription right now! <laughter> Are there key approaches that we haven’t touched on yet that we should be describing as this new field kind of evolves?

      Laura: There’s also resTORbio, which, you know, was testing a molecule that’s similar to rapamycin. And that was for respiratory tract infections in the elderly. That trial did not work when trying to get into Phase 3, but if that had replicated, that would have been one of the more — big, sort of, examples.

      There are some, sort of, drugs in the clinical, sort of, landscape today that are for metabolic disease. So things like NASH, or diabetes, or obesity — which when you overexpress these proteins in mice, make the mice live longer. So there’s this key link between things that we already are using to treat metabolic disease in the clinic, and, kind of — what might actually impact lifespan.

      Vijay: So, that’s the connection with Metformin?

      Kristen: Metformin impacts cancer deaths, too. So again, it’s like a broader aging-related mechanism.

      Vijay: Okay. That’s interesting.

      Common research approaches

      Laura: One way that we try to classify these companies is in three generations. One is focusing on traditional pathways — so things that might affect, for example, insulin signaling in the body. And those are, kind of, known targets that people are drugging with existing modalities.

      The second would be trying to screen for novel targets using platforms that are high throughput, and, kind of — either novel model organisms, or, kind of, novel in vitro or in vivo screens.

      The third would be to actually target damage directly — where you’re not saying there’s an evolved pathway that we’re knocking up or down. You’re, rather, saying there’s a set of damage accumulated, and that’s what we’re, kind of, going after in a more engineered fashion. So, you know — for example, targeting what are called senescent cells — so, cells that get, kind of, old and decrepit with age.

      Hanne: Mmhmm. The idea of zombie cells, right?

      Laura: There’s damage that builds up in the lysosome of each cell, called lipofuscin. And that is an aging-related type of damage which, when targeted, you know, may be relevant to these neuro disorders that people are, kind of, starting to work on. So there’s three different, you know — just small examples of clinical, sort of, work being done, but for age-related diseases.

      Hanne: That’s like three different frameworks, basically.

      Kristen: Well, the question, right — for the first generation of companies — is what’s the low-hanging fruit? If something is very well conserved through invertebrates up to mammals, probably it’s gonna do something in humans too. So mTOR is a very interesting target. That said, the genes that are the most important for invertebrates are probably not the most important ones for humans, right? So I think a lot of those new pathways have yet to be discovered, and will have much higher impact on longevity — phenotypes as well.

      And damage, I guess, also is sort of going directly to the major causes of disease. So I think those all make sense as approaches. I mean, it’s so unexplored now therapeutically, right? Even those drugs that have a very mild impact on longevity are, I think, going to be incredibly meaningful. I think that’s a really important consideration as well.

      Vijay: And what do you call mild? Like, 10% increase in…

      Kristen: Yeah, like a few percent increase in lifespan.

      Laura: Rapamycin is probably the most well-validated drug for extending mouse lifespan, right? But, you know, the amount of compounds that were tested to that level of scientific rigor — it’s about 30 compounds. They put 30 drugs into mice, you know — did 30 random experiments. Right? <laughter> And one of them, you know, boosted lifespan by 14%. So, I think there’s going to be tons of things that have [a] much higher effect than rapamycin.

      Vijay: Getting back to thinking about just the biology of it, it’s — is there any other trend for the “why now”? Is it just finally people like Cynthia Kenyon being brave enough to, sort of, help create the field? Are there any other, sort of, confluence of things coming in here?

      Kristen: Mapping out every single molecule in a blood sample, in a human blood sample — proteins, metabolites, whatever we can get our hands on — and seeing which of those predict living a long, healthy lifespan — and going after those that are causal. Even five years ago, really, the technologies that we’re using didn’t exist.

      Laura: Kristen, you really, kind of, changed my thinking here. When we first met, you were talking about biomarkers for longevity, and how important those were — and to be able to test our hypotheses in humans, and that’s where it all counts. And so, kind of, when you had pointed out to this was the key problem, I think that was such a big watershed for the field of — if we just make a fast, easy, cheap, reliable biomarker for aging, that’s really gonna change the whole field in a way that is more than just, kind of, getting one pathway to work it.

      Vijay: The biomarker thing is actually very interesting, because — let’s make an analogy. We have cholesterol as a biomarker for heart disease. And because there’s such a causal relationship between cholesterol and heart disease, you don’t have to run a trial waiting for people to die of heart disease. And that’s huge.

      And especially, also, you can measure it. You can see small changes go up and down. You have something that’s not binary — dead or alive. You have something that has a lot of nuance to it. And so, having biomarkers is both really useful, but — I actually think somewhat reflects just the maturation of the space, too.

      Hanne: Is there another approach where we’re all aging differently, and we need to understand things on an individual level in terms of what our aging type is? That different systems age in different ways?

      Kristen: It’s the same as with any biomarker, right? <Yeah.> Or with cancer. <Yeah.> You can, like, personalize the hell out of it, and say you’ve got these weird mutations — and therefore you’re part of this special subtype, right? And I kind of think that personalized medicine is where you go after you’ve, sort of, exhausted the things that are going to work for a broader population.

      I mean, as we discussed earlier, there are already mechanisms of aging conserved across species — you know, from yeast to us. So certainly there are also really potent mechanisms of aging that are conserved across humans. We’re focused on targeting those first, looking at the commonalities first — but certainly, you know, for certain individuals, there will be particularities to how they age that you could also, you know, treat differently in different people.

      Vijay: When we’re talking about changing paradigms, it’s not just a scientific paradigm, or even a clinical paradigm — but as a healthcare delivery paradigm as well.

      Now there is this opportunity to say, “Given that knowledge, what can we do against existing therapeutic areas — existing disease?” We don’t have to talk about “fountain of youth” — we’re talking about learning new biology. Learning new targets that can directly go into a clinical trial for a new disease. And I suspect that could be a very interesting, sort of, initial area — initial application.

      Hanne: So it’s, like, what can learning about aging actually do to make you healthier right now? In the age you’re actually in.

      Vijay: Or it can actually help you cure a disease that you have.

      Applications for treating disease

      Hanne: How — what is that connection? Can we just spell that out?

      Vijay: Yeah, well — and there’s a couple of variants of this. One variant would be an aging-related disease, like Werner’s disease — these diseases where you age rapidly. That’s kind of an obvious one, but maybe what’s less obvious is other diseases, like — could we be talking cancer? Could we be talking Alzheimer’s? What are the possibilities?

      Kristen: It’s all of those, right? I mean, age is the single biggest risk factor for those diseases. Like, 20-year-olds do not get Alzheimer’s — and we cannot cure Alzheimer’s today, and therapeutically it’s been a disaster. Everything has failed in the clinic thus far, and part of that is probably because we’re studying it in the wrong way. I mean, when we’re testing drugs in animal models, mice don’t get Alzheimer’s, and young animals do not get Alzheimer’s at all.

      Laura: Alzheimer’s disease, cancer, heart disease, and stroke — we have to study these diseases in the context of aging. And that, I think, is a new perspective.

      Vijay: If you think about just the biology of Alzheimer’s, it’s not even clear what’s going on. Like, even which protein is it? A-beta? Is it tau? Is Alzheimer’s an A-beta aggregation problem? Is it a fibril problem? Is it a tauopathy? Like, even the field can’t even agree on the biology. Even targeting a fibril, or targeting tauopathies it’s not a traditional pocket that you get a small molecule to go into.

      If you have something where the current drug design methods don’t work, it seems like applying the current drug design methods is not the right thing to do. This feels like the type of radical shift that could have an impact, and still keep us in small molecule land. When we think about this, then actually the translation part is pretty straightforward because I think the beauty of what we’re talking about here is, the current healthcare system won’t have to change.

      Hanne: Interesting.

      Vijay: That basically we have indications and, as Kristen mentioned like, not just any indications, but the biggest killers that we have to deal with.

      Hanne: Huge amount of need.

      Vijay: Huge amount of need.

      Vijay: And Alzheimer’s, where there’s at least, to date, no drug at all. I’m curious, like you could have a patient with the early signs of Alzheimer’s like, you know, with MCI, mild cognitive inhibition. Could you reverse a phenotype, or could you just delay a phenotype?

      Kristen: I think that is the whole promise and the practical approach as well. That really, if you have a drug in hand that treats aging fundamentally, it should treat several different diseases. And yes, we can work within that — the existing medical system. With the one caveat I don’t think an aging drug is going to be a great drug for metastatic cancer.

      Vijay: Yes. So stage four is probably too far.

      Kristen: Yeah. And, sort of, how far is too far? And really, these targets will probably have their most potential when they’re used in a preventative fashion. And, of course, that’s not something that the existing system can deal with. But I do think that early disease, like MCI you can at least halt progression, which would be massive, you know. And potentially reverse it with some of these mechanisms.

      Vijay: Well, and the reversals would I think, gets everyone excited.

      Kristen: Definitely.

      Vijay: But even if you could just slow down — in Alzheimer’s, slowing down could still be very, very valuable.

      Kristen: Yeah. It would still be disease-modifying. Yeah.

      Vijay: And you could have <inaudible> point against that.

      Hanne: So, it’s interesting you’re saying almost that, like, the biggest hurdle is getting the biology of aging in its approach of its own. And then once you can get the right targets, then you can, sort of, slot into the existing system and keep moving.

      Vijay: I think there’s so much about the science the biology of aging that has been validated, that now has opened the door to now treating these as targets. And actually, you know, the <inaudible> is you could, like, just identify that target, toss it over the fence to your favorite pharma and it would slot into the same type of programs that they would be running right now. It doesn’t require a radical, sort of, reenvisioning of pharma to make this happen.

      Moreover, I think you know, if you look at the history of pharma, it goes through waves of new technologies. And maybe it’s an interesting question when or if longevity becomes that hot new trend. And I suspect that in order for that to happen, you have to have one or two clinical trials that have shown this works, and then it probably just catches fire.

      Aging treatments vs. disease treatments

      I want to amplify one thing Kristen said that I think went by relatively quickly that is very, very important is that these compounds, if they are truly going after the biology of aging, will be useful in multiple indications. At first, that sounds magical, but there are actually precedents for existing compounds. So that alone is interesting that they’re already precedents.

      Hanne: Can you compare an example there?

      Vijay: I mean, some of my favorite stupid one is actually Benadryl and Unisom. So actually it’s the exact same drug. You go to the pharmacy. Often they just happen to be on opposite sides of the aisle and actually, when sold as a sleeping pill, it costs a lot more than as a… <laughter>

      Hanne: Oh, I’ve never noticed that. Is that true?

      Vijay: It’s the exact same compound, exact same dose. And if you ever take Benadryl for allergies, you get very sleepy. So that’s a simple example. There are better examples in other diseases.

      Kristen: Humira, for example, is one of the ones.

      Vijay: That’s a great example. Humira has like what, five or six indications?

      Kristen: That’s right. I think even more and, like, the world’s most valuable drug as well, right? So…

      Vijay: But this is a little different. I think in that one you just happen to like…

      Hanne: They’re similar. They’re similar diseases they’re more similar.

      Vijay: The Humira case, it’s similar diseases. In the Benadryl case, it happens to make you sleepy. <Right, right.> And it’s almost like taking advantage of the side effect. This is something fundamentally different. This is something where actually the, sort of, way to save all these diseases is to slow down aging and that’s why it has such a broad impact.

      Hanne: So, is it oversimplifying it to say aging as a kind of root cause of all these diseases or is that…

      Vijay: Or an amplifier of the diseases.

      Kristen: Or a causal driver.

      Vijay: Or a causal driver.

      Kristen: Well, look at immune aging, right? I mean, your immune system declines horribly with age. You don’t respond as well to vaccines. You’re more likely to get incredibly sick when you do get the flu or a cold, and that affects everything in your whole body that makes everything worse.

      Vijay: From a pure basing point of view, it is a causal driver.

      Kristen: There you go.

      Vijay: From just a mathematical-statistical point of view.

      Kristen: By definition.

      Vijay: And then that makes it a very natural, philosophical way to think about it.

      Laura: One of the hypotheses about why we have genetic pathways that control aging is that we’ve evolved those for a reason. That there’s a benefit to living longer enough to have kids in a different environment. And it really wouldn’t do you well to live longer and be sick, right? You want to have ways to impact all your health that pushes back all diseases. Otherwise, kind of, you just get — you know, dead [from] a different thing earlier. And so that’s, kind of, perhaps why it’d be plausible to believe that there’d be, sort of, all-disease sort of efficacy for these kinds of anti-aging therapeutics.

      Vijay: Actually, what is the evolutionary selection for aging or lack of aging? Because you could see that once you’ve given birth to children, or maybe gotten them to grandchildren — then you have no purpose, right? <laughter> I mean, you’re done from an evolutionary point of view, and you’ve — let’s say, diminished purpose from a purely, sort of, cold, evolutionary point of view — but you’re still taking up resources.

      Laura: If you have a certain fixed mortality rate year over year — if that’s actually much higher than it is today in our developed society, your probability of being dead at any one point in time in your life is actually — it gets pretty high, even independent of aging over time.

      And so, if there are any things that benefit you when you’re young that might be harmful to you older — or just, kind of, maybe things that accumulate randomly past the point at which you’re likely to be dead from other non-aging causes — they might just accumulate. And so, now that we have actually the ability to live long enough to potentially have benefited from the number of years, there’s been no selective pressure, potentially — to, kind of, live longer in that, sort of, period of life.

      Vijay: One of the things that I’m always just curious about is — what don’t we know now that we need to know? Because the problem with biology is that it’s just so complicated. Longevity and aging biology seems to be amongst the most complicated. That’s the thing that I’m always wondering about, is — what is going to be the big surprise or the big curveball, and what can we learn from it?

      Kristen: That’s a really good point, right? Because I think we’re all waiting for the first clinical trial to be successful, and that’s going to be so important for the field. So for pharma companies that traditionally don’t work in this area to really get confidence and excitement around it. But, yeah, there’s so much risk associated with bringing these first mechanisms forward and figuring out the indication path.

      I mean, you can even have a good mechanism but have, you know — defining these indications for the first time. Of course, we’re gonna get it wrong the first few times. There’s so much to figure out because it’s really such a new field.

      The current healthcare system

      Hanne: Okay. So, we’ve talked about the explosion of the field — of the study of the science, the biology of aging. And then we’ve talked a little bit about what that brings us actually right now, in terms of understanding biology and disease — but where do we meet resistance again, where we try to get this into the health system that exists today, as a kind of preventative medicine? What does that look like in terms of the end goal being a healthier life, a longer life, a longer healthspan?

      Kristen: I think that’s a great question, because you’ve got this therapy in hand — you think it’s actually slowing down aging, and, yes, you can work with the existing healthcare system and layer on indications one at a time. But really you’re not getting to the whole aging population as quickly as you can, right? And what could that path look like in the future?

      So, biomarkers is one route. I mean, maybe people are still pre-disease, but they’re frail. There’s sort of functional and molecular biomarkers that predict they’re going to be sick soon.

      Vijay: Like statins.

      Kristen: Like statins. Exactly like statins.

      Vijay: And satins will, you know — sort of, does handle a biomarker <Yep.> with the hope — when it’s done prophylactically — to avoid disease. People often say that people don’t want to pay for prevention, but we do pay for statins. There’s this old joke that plumbers have saved more lives than doctors. And that’s this point about sanitation — has just been this fundamental, sort of, floor just for human health.

      And then I think the next level up, in my mind, is getting rid of the Fritos — and no disrespect to Frito-Lay or Pepsico, <laughter> or minimizing the Fritos, you know — as much as I do like them. That’s what comes to mind.

      I mean, basically, no one should have type 2 diabetes. I mean, that’s another version of sanitation. And so now the question is — could you imagine, like, with longevity biology in hand, where you have these biomarkers, no one should have these aging-related diseases — or maybe nobody should have disease before the age of blank? And that blank goes from, like, 60 to 70 to 80 to 90 and onward?

      Hanne: That’s right.

      Vijay: Perhaps what we really just need is something to have this rock-solid biomarker that the clinicians are convinced is an issue — and then you have therapeutics that can help you manage to that biomarker. At least there’s a paradigm for that.

      Hanne: Well, well, exactly — but any therapy that really delayed aging — that really delayed the onset of disease — would save a tremendous amount of money, you know. And you can put a number on that, and you can justify a certain cost. It shouldn’t be that hard.

      Kristen: I think that’s where it comes back to “this is the decade” — because this is the first time that we’re going to see trials actually looking at all-cause mortality with therapies that are already on market today, and we’re going to see the impact of those readouts. That’s never been something that’s ever been done before. That’s truly different from any other time in history.

      Laura: And that’s the proof we need to get the system to really start recognizing it that way.

      Kristen: One would hope. <laughter> If that doesn’t move hearts and minds, what will?

      Vijay: So, that’s a great point. I’m wondering, like, what would be the analogy? Like, are we at, like — first Lipitor, kind of thing. We’re looking — because then there’s been, what? Four generations of statins since then? Before then, actually, that model didn’t even exist.

      Laura: It means you kind of form it responding to, like, the first watermark trial of the shift in paradigm — and that kind of occurring potentially as a result of these, but yeah.

      Kristen: For the field too, right? I mean, we’re now at the point where several of these hypotheses are being tested clinically. We’re going to have to wait while we really get the human proof of concept for the idea, and then once that data comes in, I think that’s going to be huge.

      Osteoporosis is a really good example, too, right? It didn’t used to be considered a disease, but there are, sort of, markers of — you know, your bones get weaker as you age, and that predisposes you to really severe outcomes and events. And now it’s recognized as one, and now there are drugs, and there’s a way forward, and payers were convinced, right? So there are case studies, I think, that we can follow

      Hanne: Where it’s kind of flipped. The understanding has flipped.

      Kristen: Exactly.

      Vijay: The mentality towards it has flipped.

      Issues around funding further research

      Hanne: Where are we in the hype cycle, would you say?

      Kristen: Yeah, aging and biotech generally, like — it’s shifted in the last few years to be a lot more accessible with, I would say, like, low upfront capital, right?

      So, first of all, the data sets that my company relies on — you know, we were for the first couple of years a data company — you know, like, for people with laptops, vastly cheaper than biology. Even if we were doing biology, now there are incubator spaces. Now there are CROs like WuXi that can do all your chemistry outsourced.

      So, I think the barrier to entry for biotech has gotten a lot lower, and really enabled a lot of these new and exciting ways to work on targets and therapies.

      Laura: And in 2011, 2013 — like, there were so few companies that, like, just having enough money to finance those companies in the space was the limiting thing. Now I think there is actually enough money, just even from the past couple of years, to fund the good ideas and the good people.

      And so when an entrepreneur comes to us and says, “Hey, I want to make,” this is a common thing, “make a lot of money and then put it back into biotech,” it’s like, no, no, no, if you’re actually a good entrepreneur, please start a company. That’s what we need more of — start a company if you want to impact the space. We lack people.

      Kristen, one thing I’m just fascinated by — as you know, you were one of the first to really go out there and do a couple of things. One will say we need biomarkers for aging, but also just build an aging company at all. I mean, there were very, very few new companies when you started. What have been the, sort of, easier and harder things that you’ve encountered as a result of that focus?

      Kristen: I mean, it’s new, right? So everyone, I think, understands that it’s riskier, I guess, than if you have, you know, another company for NASH, another company for cancer — where everybody knows exactly how that’s going to go, from discovery through validation, through your clinical trial design, through your reimbursement.

      There’s a lot of uncertainties because the space is so new, but related to that, there’s also so much opportunity. I would say that there’s more awareness now that these drugs are in trials. That there’s more — I would say — also appetite for novel mechanisms now <Yeah.> that the usual approaches are not working. So I think the landscape has changed a lot — not just, you know, at the startup level, but in terms of, like, big biotech as well.

      Vijay: Well, there’s — one, sort of, just common question for any founder serving the biopharma side — when you can do many things, what do you do first? How do you pick a therapeutic area? That’s probably one of the hardest questions an entrepreneur has to deal with.

      Kristen: Yeah, so there’s no, sort of, clear, well-trodden path — but that means that we also have the opportunity to really optimize and build something new. We’re trying to design our first clinical trials. So should it be for an age-related disease? Should it be for something closer to aging? Again, uncertainty plus opportunity, right? And trading those two things off, and making a bet.

      Laura: We’re really focused right now on just getting more people to be longevity founders. Early 2010s, it was lack of capital. Like, there was just no money in the space. Right now the big bottleneck is founders. And we’ve seen many amazing companies built by both grad students directly out of their, kind of, Ph.D — but also people coming from software engineering, managerial positions.

      And a lot of these people self-select out of the population. They say, “I can’t start a longevity company because I don’t fit the profile of a brilliant scientist founder — or a, kind of, traditional, say, investment banker type.” But, you know, they make incredible founders, and there’s just a huge population of folks out there who, I think, should be starting companies. So just to double down the idea that, like, if you want to really impact longevity, start a company. That is, like, exactly what we need right now.

      Hanne: What are the other types of founders that you tend to see coming into the field — you know, in this new field?

      Vijay: The founders in this space typically combine a couple of things. They either are biologists who have embraced, you know, machine learning or other areas — or even people that are coming from the tech side that are learning the biology. It’s a really unusual time where you can actually learn both.

      And maybe you’ve learned both from the beginning — but actually it almost feels like it’s never too late, because you can pick up both sides. But that if you can capture both sides, I think you’ll have a huge advantage.

      A nontraditional founder for us would be someone that is coming, maybe, from the pure pharma side. And we haven’t seen that yet, but I suspect they’re coming on — and Kristen’s nodding her head. And I suspect they’re coming either to be founders or as, you know, CSOs — and that they may become some of the key employees for these companies.

      Hanne: So the culture and the talent landscape [is] changing too, evolving and changing. Interesting.

      That’s it for the biology of aging. And thanks so much for joining us on “Bio Eats World.” If you’d like to hear more about all the ways biology is technology, please subscribe to the a16z bio newsletter at a16z.com/newsletter. And, of course, subscribe to “Bio Eats World” anywhere you listen to podcasts.

      • Laura Deming

      • Kristen Fortney

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      • Hanne Winarsky

      Tiktok and ‘Seeing Like an Algorithm’

      Eugene Wei and Sonal Chokshi

      In one of our special “2x” episodes of 16 Minutes (32ish minutes;) — our show where we quickly cover the headlines and tech trends, offering analysis, frameworks, explainers, and more — we cover the algorithm that powers TikTok, the short video-sharing platform that grabbed massive marketshare in cultures and markets never experienced firsthand by the engineers and designers in China, beating out other apps in the United States. Now, with talk of U.S. ownership/partnership for TikTok, what happens if the algorithm isn’t included in the deal? And what can we learn from the “creativity network effects” flywheel of TikTok; for “algorithm friendly” product design; and more broadly, about the future of video?

      The news: Given the U.S. government calling for TikTok’s business to be sold to U.S. owners last month, and several bidders coming in since, the latest news was that Oracle Corporation and Bytedance are hammering out an agreement for the former to be TikTok’s “trusted tech partner” in the U.S. This could include (as reported by Axios) their exclusive ability to oversee all tech operations for TikTok in the U.S., including access and control of U.S. user data; ability to review source code and all updates to software for security vulnerabilities; and separate boards and entities for ensuring compliance with CFIUS/ U.S. policies (and for allowing ownership stakes for Oracle, with Walmart). The deal hasn’t been approved yet [as of September 18, 2020].

      The episode: But since this show is focused on where we are on the long arc of innovation, and what’s hype/ what’s real when it comes to tech trends & the news, where does the source code (and more specifically, the “For You Page” algorithm) — which may or may not be included in the deal due to China’s revised export controls — come in? Yet it’s not just about if TikTok is really TikTok without it, or whether “the algorithm” and machine learning training data can be recreated… the real question is: How does the “creativity network effects” flywheel work between video creation and distribution — from origination to mutation to dissemination? It boils down to the idea of “algorithm friendly design”, observes Eugene Wei, who has written a series of deep dives on TikTok, and formerly led product at Hulu, Flipboard, and video at Oculus, among other things. So what does TikTok, regardless of deal outcome, suggest about the future of product development, and more broadly, the future of video? All this and more in this 2x+ long explainer episode of 16 Minutes.

      Show Notes

      • An overview of how TikTok’s algorithm came about [2:11] and the creative tools that the app offers users [5:23]
      • How TikTok encourages reusing and remixing [9:16] and serves up content based on the algorithm [14:16]
      • TikTok’s intensive tracking of user behavior [18:07] and how the algorithm sorts users into communities [22:30]
      • Issues around the possible acquisition of TikTok [27:46]
      • TikTok’s effect on commerce and video [30:00], as well as the merging of content internationally [34:50]

      Transcript

      Sonal: Hi, everyone. Welcome to “16 Minutes.” I’m Sonal, your host, and this is our show where we tease apart what’s hype and what’s real when it comes to the headlines, the tech trends, and where are on the long arc of innovation. And so, this episode is all about the short video-sharing platform TikTok (which has been in the news a lot lately), but is also about the future of entertainment and especially video. We also cover “creativity network effects” from creation to distribution, the concept of “algorithm friendly” product design, and much more.

      For those who are new to this show, I do one of these deep-dive, kind of “2x” explainer episodes — so about 32ish minutes) — every so often, where we talk about what’s in the news, but really dig into — with the top experts — the key underlying concepts. And our expert today is Eugene Wei, who has written a series of deep dives about TikTok, and formerly led product at Hulu, Flipboard, and video at Oculus, among other things. (As a reminder, none of the following should be taken as investment advice; for more important information, please see a16z.com/disclosures.)

      And for the quick news context before we go into the discussion:

      • TikTok has obviously been in the headlines with the administration calling for its sale and majority ownership of it in the U.S. last month, with multiple companies bidding since;
      • The latest news, as reported by Axios, is that Oracle and Bytedance are hammering out an agreement for the former to access and control U.S. user data; to have the ability to review source code and all updates to software for security vulnerabilities; and have independent boards for compliance;
      • But all of this is yet to be cleared by both governments.

      So, our focus in this episode will be around the evergreen and key question of where the algorithm (as if it were a single thing!) does and doesn’t come in — given talk of removing it from the equation. And more specifically, the “For You Page” algorithm, which, Eugene, you wrote about recently as quote, “the most important piece of technology” that Bytedance introduced to Tiktok, and you also called it “the hardest part,” which allowed a team of people who’ve mostly never left China to crack the cultural code and grab massive market share in places they’ve never experienced firsthand. So, what do YOU make of the news that this sale or partnership or whatever it ends up technically being, may or may not include this algorithm?

      Eugene: Yeah, I think in a lot of talk about TikTok’s algorithm (and I’m partially responsible), the dialogue’s gotten a little bit breathless, around the algorithm — it’s become like the magical MacGuffin in a film; the, you know, suitcase of whatever in “Pulp Fiction” (or something like that).

      And, while I do think the algorithm is important, I actually think that people may be overstating just like the power of the algorithm in isolation, whether it comes along in a deal or not. If you ask machine learning researchers around the world, if they think ByteDance has some algorithm that nobody has, I doubt they would agree; the algorithm is based off of very conventional research, and conventional thinking in terms of recommendations algorithms. What matters is actually the combination of the algorithm itself, and then the training data that you can train it on — and it’s the combination of the two that’s super powerful.

      But, what makes TikTok different from other spaces (like visual AI or text AI), is that there isn’t a large corpus of just publicly available training data. And so the magic of TikTok in a way is that it’s a closed-loop ecosystem: It’s an app that encourages its users to create the training data that it then trains its algorithm on. And that’s I think, the magic.

      Sonal: Can you quickly actually just walk us through the history of how TikTok actually did get that training data and then combine the algorithm to create this phenomenon where it was able to run circles around U.S. video apps, from YouTube, to Facebook, to Instagram, to Snapchat — How did they do that? Because anyone could have theoretically, you know gathered training data and come up with a different algorithm; like there’s something specific here.

      Overview of TikTok’s algorithm

      Eugene: Yeah. Well, it’s ironic because it starts with the app Musical.ly, in many ways.

      Musical.ly was a video app created by Alex and Louis, who had worked in the U.S., but were in China, and had pivoted from a short video education app. And, they launched it in both China and the U.S. — and it actually became more successful in the U.S., especially among American teenage girls, who used it to do lip sync and dance videos — then ByteDance cloned Musical.ly essentially, in China, in an app called Douyin. The irony of that is actually that the clone of Musical.ly ended up launching in a larger market, and becoming a larger app with a larger user base. And so eventually, they bought Musical.ly after its growth had stalled out in the U.S. And that’s when they rebranded Musical.ly into TikTok.

      So it’s this weird you know “multi-hop” mutation of the app that like <chuckles> — built in China; did well in the U.S.; got copied in China; and then China bought the U.S. version — it just kept hopping back and forth across the ocean.

      Sonal: Well now the hop is kind of funny because it could go the other direction <yeah>, where part of it could be divested to a sale in the U.S.!

      Eugene: Yeah… it just keeps going back and forth.

      But, all of that wouldn’t have mattered if nobody was making videos on the app, right; they actually had to build an app that made it possible for people to create a new type of video.

      Unique creation tools

      Sonal: Could you break down a little bit more into the tools? You come at this from the vantage point of someone both in *tech, and who’s also been to *film school, and is a huge lover of multimedia. What specifically — let’s talk a little bit more about what makes the tools — because frankly, there’s a lot of apps in the U.S. (like YouTube and others) who easily have the capabilities of putting these tools together.

      Now they didn’t — so that’s part of the point — but what specifically about these tools or the combination about them is really part of this flywheel?

      Eugene: Yeah. That’s where the app is a little bit underrated in terms of its creation tools. It has a really great set of camera tools; editing functions; filters that take certain high-production film techniques, and make them really accessible to a broad audience. Even licensing the music tracks was a huge thing for Musical.ly to do: Previously, if you wanted to lip sync to a pop song, you had to get like a pirated copy (or just do something that might get pulled down for copyright and trademark violations). Them signing the deals with the music labels now allow teenagers to lip-sync to the actual version of the song that they wanted to lip sync to.

      Sonal: That’s a great example of a tool that really makes something easy and fast, that was previously hard.

      Eugene: It’s two things; one is, the creation tools are really taking features and functions that traditionally you would have to use like the Adobe Creative Suite to do, on your laptop — and making it possible to do a lot of that just with your phone. That’s a huge thing because first of all, a lot of people can’t afford Adobe Suite tools, and the learning curve on them is significant; if you didn’t go to film school, you don’t know how to use After Effects. But TikTok essentially integrates those into kind of their camera suite.

      The second thing I think — and this is less about the tools — there are network effects on the creativity side, when it comes to TikTok, and that’s really underrated.

      In your podcast library, you probably have a ton of episodes that are all about all different types of network effects; the important thing to think about when it comes to this example though is just that: Does every additional creator on TikTok, make the rest of the community more creative? That’s what I mean by creativity network effects. And I actually think it’s very rare to find this form of network effect in the wild, but TikTok has achieved it, a couple ways:

      …So the hardest thing for any creator, on any app, is to just think about what to create. You know, if you are presented with a blank canvas or the blank page as a writer, can you come up with something from scratch. And the truth is, most people can’t originate ideas.

      But TikTok — because of the distribution, because of their discover page making what’s trending very salient — essentially allows you to just remix someone else’s idea. Most TikToks that people make, are actually just riffs on someone else’s idea. And so they solve that sort of blank page problem for you. You can go on TikTok and find a whole bunch of ideas, from other people.

      …The second thing is they actually structurally make it possible for you to physically riff off of the other person’s idea. So, you could do —

      Sonal: Oh you’re talking about Duets, yeah.

      Eugene: — a duet; yeah you could do a duet with someone where just like one half of the video with someone else.

      You can easily grab a component of their video to reuse in your own — like maybe you just like the music track, and the music track is the meme that you want to make; now you can just grab it, reuse it. And sometimes people upload original audio; so someone just records a TikTok video from scratch, you can even just use their audio, in your own TikTok.

      …And, the last thing is just really, I think there is a shared inspiration in the community — they make sure that if someone comes up with an inspired idea, it’s distributed really broadly. And then the sort of ethos of TikTok is that you pay it forward, everybody can borrow somebody else’s ideas.

      Remixing and reusing content

      Sonal: So, it’s really interesting because you in your original post described,“TikTok is such a fertile source for meme origination, mutation, and dissemination”.

      So we’ve talked about the origination, which is like the creative tool suite. You’re now talking about the mutation, which is this remix, taking bits and pieces — I feel like a broken record because I often talk about “combinatorial innovation”, which is such a buzzword — but it is sort of this idea of remixing bits and pieces, Lego blocks, composability in software; there’s many ways to describe this phenomenon.

      But specifically on the mutation side, it makes it very easy for people to be creators without having to be “creators”. What do you make of challenge culture within that too, and hashtags, and some of the other specifics within TikTok, that kind of make the mutation work? Because again, remix culture is nothing new; in fact, when I think of the early web, the story of it is remix culture. So like what do you think specifically about TikTok really advanced the mutation… wheel?

      Eugene: Yeah. I think that’s where the algorithm actually really comes into play — because the algorithm determines kind of who sees what. So, there’s a way in which you are incentivized to participate in certain challenges because you know the algorithm happens to be amplifying that particular meme and trend a lot right now.

      If you didn’t have the algorithm, and things had to organically find an audience, that whole challenge culture thing would work so slowly that it might not actually achieve critical mass. In a way, what TikTok is, is a mix of a free market — but also a managed economy.

      Sonal: Ooh, interesting.

      Eugene: So on the Discover page (which is a tab that you can go to), they will post what are the challenges that they’re featuring at the top: What is the hashtag; what is the you know musical track that fits with it; and what are people doing for that challenge. And you know as a creator then, that if you make something on that challenge, you have a chance to hit the top of the Discover page because it’s being featured.

      So that’s the managed economy part of it, where they actually can coordinate the entire community, and create common knowledge about what is going to be promoted. And it’s the same with hashtags, right; the hashtags that you can search on, you can see how many views each hashtag is getting right now, and try to attach yourself to the ones that have the highest velocity and momentum.

      Sonal: Right and as a quick point of contrast for those who are not as… as, in TikTok <chuckles>; in contrast, when you think about most other social networks and the trending hashtags, you actually don’t know which is more– the weighting of them at all, they could be arbitrary for all you care; <Right> it could be five people trending, it could be whatever.

      And then similarly, one of the biggest complaints people have had about YouTube is that you CAN go viral, but it’s very rare, and it’s very loaded towards very established people, as a mature established platform, because you’re essentially “gaming the algorithm.” And so what you’re kind of saying in a weird way here as you can game but not game the algorithm, <yeah> on TikTok.

      Eugene: And it does feel meritocratic in that way. You’ll sometimes click into a profile, of a creator who’s made a viral video — and you’ll see that all their other videos actually have very low view counts. They’ve sort of removed that old money effect that I describe in other social networks, where the creators who’ve been there the longest, have such an advantage over new creators.

      Sonal: Right; they’ve accrued the most “status” in that network…

      Eugene: …Exactly, exactly. So if you even see like the Meteor/Meatier pun video this week — which is about the extinction of the dinosaurs — that one was great, because she was kind of a newish creator who finally just had that first big hit.

      @lizemopetey

      is this too soon…? IB climaxximus on twitter #fyp #dinosaur ThatsHot #DinnerWithMe #MorningCheer

      ♬ original sound – Eliza Petersen

      <Sonal: Ah that’s great> And that also helps on the viewer side, right — because you’re not getting decreasing economies of scale, where the same creators videos keep getting shown to you, even if they’re no longer any good. You are always being shown stuff that they have determined, has entertained some test audience, at some you know part of the network.

      Sonal: It’s almost like evolution; it’s constantly testing for fitness <right> of this creator, essentially in this, in this model.

      Eugene: Right! We know from evolutionary theory that the stronger the fitness function or the selection pressure, the better the output on the other side.

      And I view TikTok as an “assisted evolution” ecosystem: It’s not purely leaving everything up to chance — they do put their finger on the scale sometimes in terms of hey, we have a corporate partner that wants to do this challenge; we’re going to feature it, and that’s going to give it more prominence — but for the most part, no matter how popular you are as a creator, they’re gonna let your video sink or float based on how it does with that first test audience they show it to.

      Sonal: So when you talk about assisted evolution, it’s like a combination of this managed economy and free market dynamic, which is fabulous. <yeah> Okay.

      So, so far then these are all the kind of features that now we’re kind of wrapping up on this idea of mutation. So TikTok being the most fertile source for origination with the creative tools, and, those allow some more of these creative network effects. The mutation, which allows this interaction of the community, the discovery; the fitness of creators — so you’re always getting fresh, and not only going with only the mature creators — and other kind of dynamics to play in this assisted evolution as you describe it.

      How TikTok serves up videos

      So now let’s talk about this “fertile source” for dissemination — and by the way, I don’t mean to cut these apart as if they’re three discrete things; they’re obviously on a continuum, and interact — but let’s talk about dissemination and really, distribution.

      Eugene: Yeah. So, the algorithm essentially sits at the center of all this; the algorithm is going to determine who gets shown what videos. And creators are only going to go typically, to a network where they feel like they have a chance to get disproportionate distribution of their content.

      And, the way that TikTok has sort of like short-circuited that process and accelerated it, is by using an algorithm rather than a social graph, as the primary axis of distribution.

      Sonal: Say a little bit more about what that means just for our listeners who are not in the weeds of, social networks.

      Eugene: Right. So in a typical social network, like Facebook, or Twitter, or Instagram, you start posting content, and then you try to acquire followers — and this builds out kind of a social graph, right; it’s an interconnected web of people. And based on who chooses to follow you, you will get distribution of your content to them. And then eventually if the network gets really big, they’ll put some algorithmic feed into place, where not everything you create will be shown to the people that follow you.

      I always think of this as the very traditional path of social graphs, where the follower graph kind of determines the pathways through which content travels.

      Sonal: Which is then very path dependent, shaping the future of that social network.

      Eugene: Exactly. And so, if you don’t build up enough of a following, eventually your content gets no distribution; you’ll churn out of the network, or maybe just become a viewer, where you only look at other people’s work.

      TikTok doesn’t go through that process at all. They have the ability for you to follow creators, but, that content is put into a secondary tab, the Following tab — which gets like just a fraction of the traffic that the FYP tab gets.

      Sonal: Which is the For You Page.

      Eugene: The For You Page. Essentially, they use the algorithm to determine what you see. And that just allows you to see content from people that you don’t follow, that you would enjoy otherwise. And I call this just you know TikTok basically fast-forwarding to the interest graph and bypassing the social graph.

      Traditionally, our large social networks in the West have consistently used a social graph to approximate an interest graph. But that gets them into problems.

      Sonal: Yeah… In fact, if you look at the history of original recommender algorithms, I actually met the guy who got the original patent on he used to work at Xerox PARC. And one of the things that’s fascinating about that is that he had this really cutting-edge insight [at the time] that one of the ways to recommend things is to look at your friends and find things that you like. But that’s not always true. Like, your friends’ interests do not actually capture your interest. Like, I’m your friend, and I love your views on film and you’re really into movies and books; I have those interests in common with you — but you’re also really into sports, and I have no interest in sports. And so if you were suddenly tweeting a bunch of sports things, I wouldn’t be interested in following that segment of your timeline.

      Eugene: Right, so we’ve seen this happen again and again in other social networks: On Facebook, they pivoted from, hey here’s photos from your friends, to hey here’s someone sharing like a political news story. And it’s the same on Twitter where you might follow someone who has a lot of interesting thoughts on something that you care about. But then, yes, they suddenly start posting about their favorite home sports team, or, something that you don’t care about — and then you’re stuck in this bind, because the entire feed, and the entire graph, is built off of that social following. And you start to get a higher noise to signal ratio in your feed. And that can lead to churning, or losing interest in that.

      So TikTok is like you know what, we’re not focused on that at all: We just consistently want to know what’s entertaining you right now. And we’re going to keep showing you more of it.

      Tracking user behavior

      Sonal: I’m just gonna read something from your post that’s super relevant, because you talk about how they notice everything. And if you like a video featuring video game captures, “that is noted”. If you like videos featuring puppies, “that is noted”. Like, “it is known”, it is noted, it is noted. So they notice everything basically, and they do all the work, so you don’t have to explicitly tell the algorithm by who you’re following… it just decides for you and serves things up to you.

      Eugene: The thing that’s really interesting, is that they epitomize an idea that I first read about in James Scott’s Seeing like a State.

      James Scott writes a lot about hey, you know a lot of modern governance and everything was built around this idea of, we have to make certain phenomena more legible in order for us to take actions on them. For example, if you want to tax people, if you want to conscript people, you need to actually know like how many people live in your country, what pieces of land do they operate; and so, there came about this idea of just classifying and structuring society in a way that made those units of measurement more legible, so that you could do things like tax people fairly. And we live in such a world where that’s taken for granted now that we almost don’t think about it, but if you think about a previous era, when people didn’t even have last names, it was just really hard to track your citizenry.

      I think about TikTok as an app that epitomizes the idea of “seeing like an algorithm” — where if the algorithm is going to be one of the key functions of your app, how do you design an app that allows the algorithm to see what it needs to see?

      So, the ByteDance example: They have a huge operations team that when videos are made, are tagging videos with features and attributes — so this video has a kitten in it, this video has a lion in it, this video has soldiers doing workouts in it. All those classifications actually really matter because visual AI hasn’t reached a point where you can determine exactly what the video is about. But because ByteDance invests so much in this, when they serve a video to you in TikTok, the algorithm can already see a lot of what’s in the video, it knows what the video is about.

      Next, if you look at the design of the app, what’s striking about TikTok is it only shows you one video, full screen, at a time. And whether it’s by design or accident, this is very very different from social media apps, where there are many items on the screen at one time. So with a Facebook or Twitter, if they show you like four stories on your phone screen at a time and you’re just rapidly scrolling past it, the algorithm has a hard time seeing what you feel; like, what are you even looking at on the screen?

      TikTok is different: They show you one video, one video only. And from the moment that video is on the screen, they’re looking at everything you do. And they can attribute all of that to being a clue as to your sentiment on that video. If you flip past that video, before it even finishes, that can be a negative signal. If you instead let the video loop four times, then you share it, then you heart it, then you go and follow the creator, or then you go and look at the musical track — those are all signals of interest.

      And so in that way, their feedback loop is super efficient and tightly closed. And that is, I think, a form of design that I refer to as “algorithm friendly design”. You know traditionally, all of the design principles that have guided the Valley for a long time are about minimizing user friction; in this case, they’re actually introducing a bit of friction, right.

      It would be faster if they showed me multiple thumbnails on the screen, for me to just scan through a bunch and flip through them; they’re intentionally slowing me down, and showing me one thing at a time. But in doing so they get much cleaner feedback about my sentiment — and that means that the training of the algorithm happens more quickly.

      Sonal: Oh my god, what a great explanation. So just to quickly sum up, this idea of “seeing like an algorithm” is critical. And what you really added to this as well — besides that great phrase <chuckles> — is, the fact that the product is designed to support this ability to essentially isolate the variables, in that feedback loop of what you’re studying and what you’re noticing, so that you feed it back to your users.

      Sorting users into communities

      That explains then the context that we need to know to kind of understand how the algorithm works, and what it is. So now let’s cover the third question of dissemination — and now how does that play into this whole… flywheel of these creator network effects, and then now you have distribution.

      Eugene: Yeah. So, the problem in the modern age is not that we don’t have enough content… it’s that can that content find its audience. And because TikTok has such a nice closed feedback loop — its algorithm can see what each viewer is interested in, and it can see what each video is about — it can also see how an initial test audience reacts to a video.

      It has all the components it needs to match the right video to the right viewer. And that’s the distribution part — not built on a social graph, built on an algorithm that’s just really efficient at matching content, to people who will enjoy that content. And that’s why I referred to it as “The Sorting Hat” from Harry Potter; you know more about Harry Potter than I do.

      Sonal: <laughs> I do!

      Eugene: Yeah, it’s a little mysterious how the Sorting Hat works. But it did seem to pick people with the right disposition to be a Hufflepuff, or a Gryffindor, or a Slytherin.

      You know I’m interested in really weird postmodern memes on TikTok, and it consistently serves me some really bizarre things <chuckles>; it feels like magic to me. But I know that it’s very mundane if you break it down how it works.

      Sonal: So, just to just to ground the significance of your analogy of the Sorting Hat — Imagine a world of the countless thousands, millions, billions of users out there. And then you have… this ability to essentially identify people who have like-minded kind of interests — again going back to the concept of interest graph — and sorting them into quote-“houses” of shared interest. Because in Harry Potter, the analogy is not just that these people are alike or anything, but that they have shared interests, and personality traits, or things that they like, or whatever it is.

      You know one of the interesting things about the internet, is people often talk about how it breaks down geographical barriers… going back to this idea of the Sorting Hat, the significance of this ability to distribute and sort people into houses, and communities, is really significant.

      Eugene: The thing that an algorithmic sorting allows you to do is to just scale that sorting function… infinitely. You could have editors at a magazine trying to determine what its readership is interested in, but, it will never be able to keep up with the just sheer infinite variety of its audience. You could have Reddit, which kind of sorts people into subreddits; but you still have to go and find the subreddit yourself and join.

      TikTok just allows this to happen organically, without you really having to do much that feels like work. They don’t necessarily force you through a long profiling step; you just jump in and start watching these funny videos. It’s relatively low cost; if you see a bad video or one that bores you, you just swipe past it, and immediately have a new one playing. And as that’s happening, the app is learning about your tastes.

      The other thing is people’s tastes change, over time. And so as your tastes evolve, the TikTok algorithm quickly can detect that like oh okay, this week you’re into Draco fan fiction. We’re gonna show you some more of that, because we happen to have plenty of that right now–

      Sonal: <laughs> Which you are!

      Eugene: –Yeah yeah; and I’m sure by next week, I’m going to be on to something else. <Right> So it sort of is just closely hewing to your taste profile.

      You know, Alex and Louis (who founded Musical.ly), I mean they did work in the U.S.; so it’s not like they didn’t know anything about American culture. But, the fact is that no matter how many people you have working at your company, there’s no way — if you reach hundreds of millions or even billions of users — that you can personalize, manually for all of those users. And, the algorithm here essentially says that you can scale to serve an audience of ANY size, in ANY country. And that’s really powerful.

      Sonal: So just as you made the observation earlier that the creators can evolve on this platform, and that the system evolves in identifying them and their skills as they do, so does it work for the consumers who are evolving, which is super powerful.

      I love what you said about the subreddits, too, because it’s not just the friction — actually, when you go into any kind of online community, you have to learn these norms. And here, you’re kind of immersed in a community; but, it’s actually not social at all, at the end of the day. Like TikTok, ironically, is not a social network, technically, then. <Right!> How do you kind of define it in your taxonomy of social networks?

      Eugene: I call it an entertainment network, where its primary purpose is to match these entertaining videos from creators, to the audience that would enjoy them — that’s its primary purpose. And you can obviously leave comments with creators… And a lot of creators will accept challenges, from their viewers (you can ask someone to make a video of a particular type, and sometimes in a video, they will say, “Hey, this is in response to user X, Y, Z”) —

      But you’re right, that the dominant mode of TikTok is not as a social graph. And that’s probably by design, and allows them to avoid the negative economies of scale that come from a social graph, that reaches a really large size.

      Issues around acquisition

      Sonal: Okay. Now let’s bring it back to the news and the trends; so this show is about covering the long arc of tech trends — we’ve talked about the evolution of recommender systems, the social networks, we’ll talk about video in a second — you’ve started to tease apart what’s hype, what’s real (including some of the hype you yourself may have put out about the importance of the algorithm;) —

      To close the loop on bringing it back to the news, where do you stand on this idea, if in the final agreement — and again, who knows what’s going to happen ‘cause this changes every day — the algorithm is or isn’t part of it? Cuz China just updated their export controls to be able to refute the deal if they don’t want it to be in there, the source code. How much of a difference do you think it makes? Do you think if they were to back engineer an algorithm that functions similarly, that noticed everything, given the current product design — do you think they could conceivably still recreate that sort of wheel, given that there is already this critical mass of users on TikTok?

      Eugene: Well, earlier, I talked about how I think people are maybe overrating the algorithm in terms of just like you know how unique the algorithm is itself. But: It is certainly true that if you purchase TikTok and it didn’t come with the algorithm, it would take you some amount of time — even if you had all the user data, video metadata, all of that — to sort of rebuild, and retrain, an algorithm of your own.

      There’s always a risk with a social network that in that interim period (maybe it takes you months, maybe it takes you a year), that people would find that the app wasn’t as responsive… to their interests anymore, and that they might churn off of it. So, certainly you would rather have access to the full closed loop that allows that information to be fed back cleanly into the algorithm.

      The algorithm’s already been trained; the hardest part often with a lot of these algorithms is getting that training data set, and they already have just a massive training data set of these videos with I don’t know, a gazillion hours of view time. You have a lot of users whose tastes are — have already been profiled.

      So, yeah. I would say that it is possible to rebuild an algorithm. I think with the right tech companies, you have a lot of the talent here in the U.S. that can do that. But, that process takes time, and that’s risky.

      Broader effects of TikTok’s approach

      Sonal: Okay, so now I’m going to ask you just two last quick questions on sort of the long arc of tech trends, and then one practical question before we switch to that.

      As someone who thinks a lot about product, and multimedia, and you know has worked on designing– you’ve actually actively designed many of these things in production, do you have any advice, or what are the implications, of all this — besides the fact that this phenomenon could occur, penetrate into mass market — what do you think about how this affects your thinking for finding product-market fit, or designing products in this… kind of era?

      Eugene: Yeah. You know, I think a lot of people have said wow there hasn’t been any big new social network in recent years other than Snapchat, that have come up to challenge Facebook, Instagram, Twitter, those giants. So I think one big learning from TikTok is, hey, there’s an alternative approach that might work — which is to just cut straight to the interest graph.

      And… that the way to do that would be to figure out, can you design an experience, a user experience, that allows a machine learning algorithm to get access to a unique set of training data. And I think it is probably possible in other fields and disciplines. I do think it takes a new approach to design, which is this “algorithm-friendly design”.

      Sonal: Yeah. “Seeing like an algorithm”.

      Eugene: Yeah, exactly. You’re like hey, this algorithm isn’t sitting in this design meeting with us right now; but it’s really important that when we’re thinking about what does the UI look like, what are the feedback loops, that we’re capturing the right data for the algorithm to be able to SEE, and do its work.

      So I think that is a novel new sort of design- and product-development paradigm, which TikTok has created. (And you know really ByteDance even used that to develop their first trendy hot news app in China called Toutiao.)

      Sonal: Okay. So then now arcing back up a bit to broad trends, how do you view this in the long arc of innovation when it comes to video, and the future of video? Because one of the recurring themes of your post — which it was kind of a recurring motif — is, we really haven’t figured out video; we’re actually still at the beginning of video; there’s a lot more to be done in video; it’s shocking to me how little people are doing video well. What are your high-level takeaways on that front when it comes to that tech trend and evolution?

      Eugene: Two big takeaways. One is, that I think we consistently underrate the degree to which people respond more broadly to video than they do to, for example, text. You know the number of people who are going to read books, all the time, is just a fraction of the number of people who enjoy watching video.

      And so, that really matters at scale. When you’re talking about reaching a broader audience. I don’t think we have a medium that can challenge video, in the world. I think the evidence is overwhelming. The second thing is…

      Sonal: I mean, I would give you a little bit on audio <Eugene laughs> but we don’t have to go off on that side tangent. Let’s just stick to video <right> Keep going! The second thing.

      Eugene: The second thing… is that in order for video to scale as a medium, you do have to do some work to overcome some of the challenges inherent to video. Video is traditionally a little bit harder to scan for conceptual information; you know, it’s harder to understand what’s in a video. Even if you’re watching a video, if someone sends you a video, sometimes people are like, I wish you would just send me the transcript so I can just scan through it really quickly. You know scanning video is even hard.

      So, TikTok fortunately, the video is all really short. And they allow additional layers of metadata; you can bring text into the video, really easily. And so video overall as a medium, is a richer medium on TikTok. If you can bring that all to bear, then I think video becomes more relevant in other fields — like, education, or you know if you want to pick a place to go on vacation, or you want to pick a restaurant to go eat at.

      Sonal: Yeah. Our partner Connie Chan’s actually argued a lot about the power of every commerce will become video, and every video will become commerce, and sort of the intersection of the two.

      Eugene: Right… video is really just the bed for a whole bunch of other information to be laid on top of it.

      Video is just such a high bandwidth medium. I think we haven’t really taken advantage of that full level of bandwidth in the past. We know that humans are super attuned to body language, to reading another person’s face; you know, one of the downsides of trying to read body language over Zoom, you may have like 15 people in a Zoom, each is just a small thumbnail; you can’t really see anything, other than a blurry version of their face. There is something that is lost when you lower the bandwidth. And video brings that back, and video gets higher fidelity every day. And, you know something like TikTok now is just making more use of that full bandwidth.

      Sharing content internationally

      Sonal: Great. So then the last question: What do you make of this larger phenomenon, given that the whole point of your post is about how this is the first time a social network from another place has really cracked into a different market. (And we haven’t even talked about India, and Middle East, but it’s also cracked into other markets, not just the U.S.)

      The thing that fascinated me about your post, is this idea that there could be this internet layer that crosses regions and cultures. And you share an anecdote at the end of your post where the engineers that you — the office that you visited, they had like all these Hindi lyrics and Bollywood lip synching going on, and not a single person in the office even knew what they were seeing, or could even read Hindi. That is kind of amazing…

      Eugene: Right. And that’s the one powerful thing about video; a lot of it doesn’t require you to understand the language. In fact you know a dance video, a little skit, even if they’re speaking in it, often you can just interpret based on what’s happening on screen <yeah> what they’re talking about.

      That language is international. In a way, it’s more international language than even text. You know a lot of people in America still can’t read a lick of Chinese <mhm>, and a lot of people in China can’t really read English. But when it comes to video, and you show somebody a video on your phone, everybody can understand you know, oh this is a cute baby video, or this is an animal doing something funny. Netflix, for example right, is trying to figure out, hey which shows that we make in one market could carry over to other markets; if we can, we prefer that because it makes our content spend more efficient.

      Sonal: All right. So, Eugene, bottom-line it for me. A lot to say, but on this explainer/news commentary episode on to-algorithm or not to see-like-an-algorithm, what is your takeaway on the news, bottom line it for me.

      Eugene: Look, I don’t know what’s going to happen with this deal… regardless of that, I think TikTok’s impact will last, in that it provides a model, for how in an age of you know increased use of machine learning algorithms, you might build a new sort of network — that’s really built around algorithmic recommendations, and that shortcuts you to building out the interest graph.

      Which ultimately, is probably one of the most valuable graphs in the world. If you think about how social networks make money — trying to determine which ads are relevant to serve to you; on the other side, the advertisers want their ads to reach the right audience — that’s ALL interest graph; that’s not really social graph. And so TikTok came along at a time when everybody was like, well, we’re stuck with these social networks. And they kind of snuck up on everybody from the side. And that’s a remarkable story.

      Sonal: Thank you so much for joining this segment of “16 Minutes”, Eugene.

      Eugene: Thanks for having me.

      image: Eliza Petersen

      • Eugene Wei

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Reining in Complexity — Data Science & Future of AI/ML Businesses

      Peter Wang and Martin Casado

      There is no spoon. Or rather, “There is no such thing as ‘data’, there’s just frozen models”, argues Peter Wang, the co-founder and CEO of Anaconda — who also created the PyData conferences and grew the early data science community there, while on the frontlines of trying to make Python useful for business analytics. He views both models and data as fluid, more like metaphysics than typical data management… Or perhaps it’s that when it comes to data, those with a physics background just better appreciate the mind-bending complexity and challenges of reining in the natural world, and therefore get the unique challenges of AI/ML development, observes a16z general partner Martin Casado — whose first job after college involved computational physics simulation and high-performance computing in Python at Lawrence Livermore National Laboratory. (Wang, meanwhile, graduated in physics.)

      But this not just a philosophical question — the answer has real implications for the margins, organizational structures, and building of AI/ML businesses. Especially as we’re in a tricky time of transition, where customers don’t even know what they’re asking for, yet are looking for AI/ML help or know it’s the future. So what does this all mean for the software value chain; for open source collaboration and commodification; and for the future of software businesses? After all, it’s not written in stone that “All information systems must be deconstructed into hardware, and software, and data” and that “software must have these margins”… Will there be a new type of company?

       

      image: Pawel Loj / Wikimedia Commons

      Show Notes

      • Discussion of various data management tools [1:44] and whether new tools are needed [5:52]
      • Software vs. hardware [10:00] and a discussion of what data is [13:04]
      • Managing the inherent complexities in data [14:22] and the backgrounds of the hosts [16:47]
      • Different company types that are trying to rein in data complexity [22:00], and a vision of a new company built on AI/ML workflows [32:17]
      • Advice for companies in the AI/ML space [38:37]

      Transcript

      Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. For this week’s episode, we have one of our hallway-style conversations. And this one is literally like eavesdropping in on a debate and discussion that actually started as a Twitter thread debate and discussion — all around the question of whether and how data and AI/ML (machine learning) companies are different than software companies, and what that means for the future of software businesses. Our guest even questions our view of software eating the world — or rather, asks what happens when software is everywhere? What comes next? 

      Our guest is Peter Wang, the co-founder and CEO of Anaconda, who also leads our Open Source and Community Innovation group — as well as created the PyData community and conferences, and has devoted a lot of time and energy to growing the data science community there. And he’s in conversation with a16z general partner Martin Casado, who’s written a lot about the evolution of software businesses, the new age of data, and especially AI/ML economics. You can find those pieces at a16z.com/mleconomics.

      The two dive into a number of themes throughout this conversation, ranging from open source and crowdsource innovation, and the messy ways that innovation really plays out — to what it means when you move from hardware to software to data and AI/ML — abstracting something that is not just complicated, but actually complex. And then, they touch briefly on what it means practically in building a new type of company, as well as the evolving role of data scientists. But the conversation begins with their shared vantage points in coming from physics, which is relevant here since these new kinds of businesses and products involve a process of experimenting, much as with physics.

      The best tools to run AI/ML

      Martin: Both you and I come from the physics, computational physics background, and we both, kind of, been pushed into this data, AI/ML data science — and I don’t know if that is coincidence, or if we have an affinity for that. Before we get into that, though, there’s  kind of a competing view of the world, which basically says, “SQL can do everything.” And it’s funny, we spent a lot of time actually looking at the data science, or the data landscape, and it feels like there’s two kinds of worlds. There’s, like, the data warehouse maximalists, which like — will stick all data in the data warehouse, and then we’re gonna do SQL. And then we’re gonna have some extensions to SQL, like you see popping up in, like, BigQuery, or whatever, and that can do everything that needs to be done. And oh, by the way — if someone’s using Python and R, all they’re really doing is basic regressions. And so we can just make that a simple extension, and we’re done.

      And then there’s the other view of the world, which I like to call the Hadoop refugees, which is like — actually, we do hardcore computation, and we need R and Python because the stuff we do is very sophisticated. I mean, I know you’re squarely on one side of those. But I wonder, like, do you think there’s a convergence that happens? Do these stay two worlds? Does one become irrelevant? Like, what happens there?

      Peter: Just because you oppose extremism doesn’t make you an extremist, right? I would say data warehouse maximalists are extremists.

      Martin: <laughter> Fair enough. Yeah.

      Peter: And I see a heterogeneous world. It’s the old yarn about, I guess, I don’t know — there’s so many variants of this. But Alan Perlis, a great computer scientist, has some really great quotes about — some irreverancies about these kinds of things. But I would say that to the idea that everything can be expressed in SQL, it’s like — which SQL? With how many extensions? Because at the end of the day — and with how many like extensions upon extensions, and Multicorn on your Postgre actually running a Python kernel. Yeah. I guess you’re doing a SQL, but you’re running a Python script, you know, so that’s not really — it doesn’t count. 

      And frankly, a lot of stuff runs Access and VBA in this world. VBA isn’t SQL. I think if you choose to look at the world through a particular lens, you can choose to count everything else as residuals and rounding errors, but if you take off those lenses, you see a much more diverse landscape. And I think that’s where, for me, I see the space for SQL, and I understand the reasons why — it has evolved into a particular kind of animal. Like the shark is still the best predatory fish in the ocean, but it’s not the biggest predator in the world.

      And I think there’s something about that, that if you’re in the ocean, you’re gonna basically [be] shark-like if you’re gonna eat a lot of fish. So if you’re in that business data analytics world, especially because a lot of business data looks like fish — it’s evolved to look like food for the sharks. So that’s kind of the way it is. But what Hadoop opened up back in 2012 — I called it the Hadoop battering ram. I said, “Listen, we’re not gonna win the Hadoop game. We’ll let the Hadoop vendors go and fight against the Teradatas, and the Oracles, and all the classical data warehouse guys. Let them do that thing. Once it battered down the door, we’re gonna come flooding in with all sorts of heterogeneous approaches to data science, data analytics — things that are hard to ask in SQL.”

      And moreover, there’s a term I use, which I don’t hear used very often. Now, obviously, you’ve heard the term shadow IT, which is used quite a bit, but there’s a shadow data management — that’s a far, far more insidious and dangerous problem. When I was at a large investment bank, they had a million-dollar Oracle database sitting somewhere, and it was too slow to actually run the analytics they needed. And so what they did is they had an instance of this Oracle database, it costs a million bucks, and what they did is the only query they ran was a bit full table dump into a CSV. And then they took that CSV, and they did everything else with it. And it was Python scripts. It was some random Java crap. It was a bunch of other stuff. And it was sort of like, so if you’re a data manager — if you’re, like, in the data management practice, you say, “Wow, we just have another big old million-dollar instance stood up. Our data management techniques are great.” It’s a, what do you call it, a Potemkin village, I guess, right? But then when you actually go, and you ask the developers, “Hey, where’s the source data for this stuff? Where’s prod data coming from?” Like, “Oh, yeah, this file share backslash-backslash something or the other, or you know, that file.” I’m like, “That file? What about that database?” “Don’t touch the database. It’s too brittle.” Right?

      So there’s this kind of stuff going on, and everybody listening to this knows what I’m talking about. That shadow data management is absolutely a pernicious problem, and data science is just eating it alive. Because to ask the question you want to ask, you have to integrate datasets together. Master data management is about siloization, normalization, and all this kind of stuff.

      Martin: You’ve hit to the segue,, too. I just think it is so germane to what we’re here to talk about, which is — there’s clearly problem domains, which SQL is totally fine for, right?

      Peter: Yep.

      Martin: And you can only get the problem domains, which is just not good. I mean, like any sort of hardcore statistics is just not very good for. And the point of us being on this podcast is actually to talk about, okay, like, “Listen, we’re seeing kind of new types of companies and new types of workloads, and they’re around kind of processing data.” And I totally hear you, that this shadow data management is a real issue. And you can make an argument why that exists is not because people are stupid, or they don’t know how to do good workflows. It’s like, literally, we don’t have the tooling to deal with data in the right way. 

      One macro question that I have that I would love to hash out with you is, are we seeing a fundamental shift in workload that requires a fundamentally new set of tools and a fundamentally new type of company? Or is this just more of a transition where we can kind of put into service the old tools? And I just want to be a little bit more specific, which is — in the past, you had your toolkit of systems approaches, and you have a software system, and you’d kind of pull them out and apply it to the problem, and SQL is one of them. And we kind of understood how those software systems behaved. And we kind of understood how the company is supposed to run and behave.

      You know, as an investor looking at a lot of data companies, they just don’t look the same. The types of tools they use, the type of operational practice they use. And the one that you pointed out was a great one, which is — now data becomes so primitive that you want to actually apply, like, software techniques to, in a way, but we don’t have the tools to do that. And then we’ve written posts about [how] margin structures look a lot different, the way you build your company [is] different. And so, just — do you think this mess is because data scientists don’t have formal CS trainings? Or do you think this is an entirely different problem domain, and we should actually look at what the future looks like for that, and development tools, etc.? This is like the heart of what we’re talking about.

      Peter: This is absolutely the heart. And I will try to start from the top, which is this concept that every baby or every child is born — and they’re raising it, they think their child is normal, right? They think of, like, your child is like the normal thing. So you have developers coming online in the late 2000s, let’s say, and they think this is the world. Even me as a professional starting in ’99, right, it’s like, “Well, this is just what there is.” The more you start researching history and looking back, you’re like, you know what? We’re just building in this industry — we just layer. It’s frozen accident, on top of frozen accident, on top of frozen accident. Very, very few times do people make principled intentional revolutionary shifts, right?

      Martin: Totally. Yeah.

      Peter: You basically Band-Aid a substrate. Okay? So starting from the top, what I would say is that, there is no law — there was nothing carved in stone that Moses brought down from the mountain that said, “All information systems must be deconstructed into hardware and software and data.” There’s no such thing. It was information systems, full stop. The fact that we had different cost structures for innovation in hardware, versus software, versus networking, and so forth — that has led to different rates of innovation, different places, things like that. And so when a business steps in and says, “Okay. What’s on the shelf that I can use to accelerate my business processes?” Then it makes sense, because this thing, that thing, the other. Like, when you buy a car, you buy the car, and then you put CDs in the car. You don’t go buy a car with a CD pre-spec, right?

      Martin: Is there the exception of technical innovation in certain areas? So for example, like — we now know how to build systems that extract very useful information out of data pretty simply. That didn’t really work in the late ’90s. Like I remember, the whole first neural network, like, genetic programming.

      Peter: Oh, yeah. Right. Right. Yeah. Yep.

      Martin: The asthma of the late 90s. I did a number of projects on that that didn’t really work. They actually work now. So you could also argue that the technical landscape [has] changed. It’s not just been a macroeconomic issue on the company.

      Peter: Yeah. I mean, ornithopters work if you can flap hard enough, right? It doesn’t necessarily mean it’s the right architecture. <laughter> And it depends on the density of air. Ornithopters might work great [on] Mars, but not on Earth. Right? Propellers work better on Earth. Right? Well, with internal combustion engines, and etc., etc. But the point is that, yes, you’re right. I guess my point could be said thusly. There is a multi-dimensional optimization surface we should be thinking about, not just the optimization surface of software, or a data architecture, or data management, and things like that. I mean, yes, someone did software-defined networking, and you know that better than anybody.

      Software vs. hardware

      Martin: But here’s what’s interesting to me, which is if you build a hardware company, the tools you use, the money that you need to raise, the innovation pace, is defined one way. And if you do a software company, it’s actually defined quite differently. Although you still use, like, a lot of the same practices, it’s still engineering. You can still modularize. It’s not clear to me that as soon as you move to data, you’re in the same domain. 

      Software, to me, feels like an engineering problem that you can modularize — you can build interfaces, you’re building it from the ground up, you control all the primitives. Data feels like science. It’s like you’re trying to reign [in] the complexity of the physical world. Right? It’s one thing to, like, build a house — building a very complex building is very hard, and we had to do all this design practice and the other, but we got the skyscraper. That’s very different than understanding the cosmos, because the cosmos is so complex, and you don’t understand what it is, and you don’t have a blueprint. And data companies are defining the cosmos more than building the skyscrapers. Does that make sense?

      Peter: You hit it on its head. I’ll just back up and comment on one thing relative to the hardware and software. Hardware is frozen software to some extent, but the pace of — oh, how to put it? Because hardware is expensive and slow, and has been, at least historically — the industry has a much more robust view towards standards. Now here’s the thing — because you have standards, now you have a binary, bullshit proof, “Does it work, or does it not work?” kind of thing. Okay. That then reflects and changes, then, kind of what you need to do. 

      Software — what it does, it makes mistakes in hardware expensive, because there is an inter-subjective reality beyond any particular vendor about what is a mistake. In software, because it moves so fast, it’s too fast running to build specs and hard specs and say, “Did you meet this performance spec you said we’re gonna do?” No one cares about that. Software is just so fast and loose. It’s like jazz. I mean, so — because it moves fast. And there’s not a — you can’t put that thing in. Then the price of making a mistake in software is almost completely subsumed or lost. And so it’s cheap to make mistakes in software because the cost is invisible.

      Martin: 100%. However, the actual engineering practices aren’t that different, as far as, like — I mean, you’re absolutely right, like, formal verification is much more important in hardware, but it still feels like engineering to me. You know exactly where you’re going. You have a roadmap. You build an engineering team around that. Data is different.

      Peter: Data is different.

      Martin: You don’t have a roadmap. Like it is the universe that you’re trying to like, you just infect inside out.

      Peter: In fact, this is the exact critique. You’re absolutely right. When you talk about what you do in software and hardware companies, you are trying to manage complexity, for the most part. You get something, but the thing that always screws you — I figure, every kind of engineering is trying to achieve some kind of lift while finding some kind of drag. Right? And in the case of software or hardware engineering, usually, it’s achieving performance or something like that, or some scale of computation, while minimizing complexity — and having manageable errors and things like that. Okay. So that’s those things. But it’s very goal-oriented.

      Martin: Yeah. Building to a goal. It’s one thing to say, like, “I’m gonna build this complex system, which you can basically describe, do mock-ups for any destination.” That’s very different than saying, “Extract insight out of this.”

      Peter: That’s right. The great John Tukey said, “There’s two kinds of data analysis. There’s confirmatory — kind of, reporting mode — and there’s exploratory.” And the thing you’re talking about, the reason why data smells — and data practices smell like science, is because there is no such thing as data. All data is just frozen models. Right?

      Martin: Totally. 100%.

      Peter: Every single data set comes from a sensor, even a picture. Everyone thinks, “Oh, well, I took a picture.” Right? That’s just raw data. No, it’s not. There’s a Bayer matrix. There’s a log transform. There’s a gamma correction. And, fundamentally, there’s an exposure time, which is a temporal sampling domain. So there’s all of these things. There is no such thing as data. There’s just frozen models. And where businesses get screwed up is when they treat data management as, sort of, this goal-oriented siloization — it’s a static artifact, and it is artifact management. It’s almost like a — sort of ad hoc library process. And that’s not the same as the kind of data thinking — or the way when you think about data in an ML/AI sort of world. Because in that world, we see that models and data are both fluid. It’s much more from a meta — not to get too metaphysical, but it’s more of a process-oriented metaphysics. It’s much more temporal-oriented than the static views that current data management practice has. And that’s why I think the SQL database extremists are not going to win this particular round.

      Martin: So, I’m a systems guy. Right? Like, I did my Ph.D in computer systems. And in systems, we have five tricks. It’s like virtualization, caching, you know — like, we literally have five or six tricks that we throw at every single problem. And you can build amazingly complex systems with these things. Like, you know, we understand distribution, we understand consensus. And so while a piece of software like Google is very complex, it actually can be reduced into subproblems that we know answers to, and then you know, we can — so I would say, like, the relative complexity — the relative entropy of a software system — is finite. It’s not clear to me if you’re trying to use data to run a system that the entropy is as finite.

      Peter: Well, yeah.

      Martin: Meaning you don’t control nature. I mean, what do we use data for? We use data for pricing. We use data for fraud detection. We use data for calculating wait times. Okay. So what are the inputs from these things? These things — it’s like people’s behavior. Like there’s so much entropy in all of us. It’s like the weather. It’s like this…

      Peter: It’s hugely lossy, right?

      Martin: Well, it’s these classically chaotic, high entropy systems. And so one of my theses is — and I’ll just have to test this on you, is that building a software system is a relatively low entropy exercise because you’re dealing with primitives that you understand and you’re engineering it. Where actually trying to deal with data, you’re reining in so much entropy, and you’re trying to extract it. That ultimately is why we end up with different companies, because it’s just much, much harder to, like, deal with that much complexity.

      Peter: Yeah. Well, that makes a lot of sense. And the Cynefin framework talks about the difference between complex and complicated and chaotic. Right?

      Martin: Yes. Yes. Yes. Sure.

      Peter: Right. And so complicated. And I think the pithiest way to say this is —  complicated means that you can take it apart, understand the bits, and put it back together again. Complex means that you cannot do that. Right? So a fine Swiss watch is complicated. A cockroach is complex. And so I think when you talk about computer systems — because I’m not a systems guy like you are — but one of the best things that I’ve heard about it is that everyone thinks — what is the quote? Everyone thinks distributed computing is about space, but really it’s about time. What is the time horizon in which we can define a unit of atomicity? What is the time to coherence? Right, etc., etc. And so it’s always a space-time trade-off.

      And I’m sorry, I’m making this look so like, into the physics world, but I see it that way because it’s a natural flex for me. In fact, I wanted to major in computer science, but my dad — who was a physicist — he said, “Look, son, if you become a computer programmer, if you go into computer science, you’re gonna become a programmer, and you’re just gonna build tools. If you’re a scientist, though, you’re gonna be the one using those tools to make an impact.”

      So I majored in physics. But then, as soon as I got out of physics, it was ’99. And I’m like, “All my friends are getting, like — they’re getting starting bonuses, and they’re getting jobs, and they’re worse programmers than me.” And so I ended up joining a computer graphics startup. And that’s when I started using Python, was in ’99. I realized that I could script a bunch of C++ much better with Python than with broken template support in Visual Studios. It was God awful.

      Martin: I came to networking by way of computational physics. Actually, when I was a computational physicist — I was a computer scientist doing computational simulation at Lawrence Livermore National Lab. That was my first job after undergrad. I was a huge Numeric user, because that was the only way to do high-performance computing in Python — and from what I understand, that became Anaconda. I would love it if you would kind of give the history of that project.

      History of programming tools

      Peter: So in ’99, it was Jim Hugunin — and I think there’s some others that I might be forgetting — can be credited with working on some of the early matrix stuff. And then Jim Hugunin worked on Numeric, and they realized that the operator overloading in Python would allow you to do something that looked a bit like MATLAB. You know, like, it’s okay — it looks like you’re right back to code. And it’s like, “Hey, this hack kind of works.” And also, Python’s C level extensibility meant that they could build a little tight C library that would be fast. So you’re writing the scripting thing, that little syntax looked like MATLAB, but it ran at basically C speed, which is really important.

      So then, it turns out, though, that some of the features they built — the Space Telescope Science Institute folks, the ones who run the Hubble telescope — they had some other ideas about what they wanna do with this library, and Numeric wasn’t quite flexible enough, or some other stuff. But they created an alternative matrix library called NumArray. And NumArray had, like, fancy indexing. NumArray had a few other things. And so the ecosystem in the early 2000s — when I first got my first paid job doing Python, [it] was 2004 and I was doing consulting on Python, and SciPy, and all that stuff. And there was still a split between NumArray and Numeric. Or, in fact, most of the libraries that were trying to build on top of this stuff — they built a compatibility layer called Numerix, which would then flexibly import sub-symbols from these different libraries depending on what you’re trying to — it was terrible.

      Martin: The wild and wooly days of early Python.

      Peter: You know, it’s a mess. Crowdsource innovation is always a mess, but the result is still nice, because what happens is you end up getting somebody like Travis Oliphant — who comes along in 2005 and says, “This is a mess, and this is slowing down innovation because everyone has to do the work twice. We got to make it work with NumArray and with Numeric, and we can’t make forward progress.” So he spent a year of his life into making — just coding and designing, and he made a really nice thing, and he called it NumPy.

      And he came out with it in, like, end of 2005, 2006 timeframe. And then the world rejoiced. And I was like, “Oh my God, this is great. This is the unification we needed.” You know, at the SciPy conference in Pasadena the following year, we gave him an award. Anyway, that’s what happened in the mid-2000s. And then, many years later, then in [the] 2010 timeframe, he actually joined the company I was at, Enthought, and then we had many happy days there, doing a lot of scientific computing, consulting. Which is fun for science nerds, but a niche area. Right?

      But then we started getting contracts and consulting inquiries from hedge funds, and from banks, and investment banks, and things like that. And by the end of the 2000s, I’m walking to the floor of, like, JPMorgan, Bank of America, and they have thousands of people relying on SciPy and NumPy to run advanced models. You had coders sitting next to traders, like on the energy desk, and you’re like, “This guy is asking me really deep questions about SciPy. He’s really trying to do stuff with this.” So I had this insight that — I think Python is ready to go into the mainstream, like, business analytic space. And that’s not just MATLAB that it could be taking market share from, but maybe SAS. So at the same time big data was starting to crest at that time, or peak — and I realized that people wanna do more than just ask SQL questions of their big data. And in fact, when I went to the first Strata, in 2011, all of the vendors on the show floor were selling many different flavors of Hadoop. SQL integrations, faster Hadoop, etc., etc.

      But then, when you go to the tutorials, every single data science tutorial was teaching Python and R, but there’s no Python vendor. And also, Python is kind of janky for some of the stuff. It doesn’t play with Java very well. Python and R were both second class citizens in the Hadoop world. So I said, “You know, I think there’s something here.” And that’s why I started the company. We started as Continuum Analytics in 2012. And it was Python for business data analytics, Python for data science. That’s what led to that. Anyway, that was a long, sort of, exposition. But to your question about the history of all of this — how this came [about] — but I think that when you talk about software systems, it’s actually very interesting. We build software systems thinking they’re merely Lego bricks — that we make relatively homogenous, or homogeneous. Or, well-structured studs are spaced this way, they’re this big and this tall — and then we can stack them together, and boom, now you have a bigger Lego.

      But in reality, when you look at any real software language in modern software systems, there’s complexity to it — more than the complication. And that’s where your worst bugs lie. You know, like, you have some npm module that pulls in some other crap, and that interferes with some other crap, and it tries to install this other thing on your system — and now you have complexity beyond the complication. So I think the practice of software is bedeviled by the fact that it actually is playing, at this point, with so much complication that it basically appears complex to our human minds.

      Tactics for dealing with data complexity

      Martin: Barbara Liskov has my favorite Turing Award acceptance speech ever, and if you haven’t heard it, you have to hear it. And it’s basically about modularity and computer science. And it’s how you can take big problems and make them small problems. Like engineering with modularity — you can rein in complexity. So you have a complicated system, but I think you can actually manage the complexity. I’ll give you an example on the data side where that’s not the case. There are natural systems that are self-similar. By self-similar, it means that they retain the same stochastic properties no matter what zoom level.

      So, unlike a software system, if you’ve reduced it down to a method, you’ve got, you know, a fairly simple abstraction. There are some natural systems like, say, coastlines — that it doesn’t matter at what level you look at it, they still are, like, super complex. So one thesis is like, yes, software systems can be complex, but like, they’re more complicated in that you can modularize and focus on things. That’s not necessarily the case with data. Data is as complex as the natural world. Again, like, you don’t have control over the weather, and the weather is self-similar. And no matter what zoom level you look at it, it still maintains the same stochastic problems. It’s not like data. You don’t have the tools necessarily to reduce the complexity to something that is merely complicated like you do with software.

      Peter: Right. So the question then in the data practice world, then — let’s just keep it at that level, then, which I think is a great place to be talking about it — to which point do you stop? What is your optimization criterion? Right, because all engineering is a trade-off. So for the amount of effort you want to put in, how well do you need to understand that coastline? If you’re trying to target a guided missile into a window of a building, you don’t need to map the coastline down to a millimeter, right? So on and so forth. So I think that when you get to data, you recognize that, it really, ultimately — if you actually want to get all the value out of it, you’ve got to loop it around into the overall OODA loop of your business — the observe, orient, decide, act loop —and actually take action with it and correct and zoom into the appropriate level.

      Real-world implications for businesses

      Martin: I think this is kind of what this all boils down to. So now the question is, let’s say that you’re building a company — that instead of the goal of the company is building a modular software system, [it’s] reining the complexity of data, which we’re seeing more and more companies do. What does that mean to deal with that much complexity? So what you just mentioned is, well, okay, maybe you look at, like, the different zoom level — or maybe you’ve got, like, a full feedback system, or whatever. But before we even get to how you do this, I would like to either agree or disagree that the companies trying to rein in that complexity are different.

      Peter: I completely agree with that. The companies that actually understand even the problem they need to solve, they have a better chance of solving the problem. Because it’s actually very much like cloud computing. It used to be — how do I build the software on the basis of the computational resource I have access to? Well, once you have ability to access essentially limitless computation, you’ve got to ask about, “Well, what is it I would need to build? What do I really wanna do, right?” So I think with data, it’s a similar thing, where you say, “Well, you can put in for any <inaudible>. You can put in more money and get more texture, more resolution on your predictions.”

      Martin: Exactly.

      Peter: Where do you stop?

      Martin: Exactly. Exactly. Right.

      Peter: And stop is, like — I can only convince this CEO to hire three data scientists? So that’s where we stop? Is this what three scientists can do? I think that’s how a lot of people are winging it right now, but the interesting thing with the hedge funds — you look at them is — they understand this. Like some people say, “You know what, we’re not gonna work at the microstructure level. We’re just not gonna do that because there’s a few big players that play the high-frequency stuff. We’re gonna leave that out. We’re gonna do kind of longer-term stuff and do bigger strategies — some, you know, longer-term strategies.” So they self-select into zones where they believe they have the observational capacity and connect that to execution capacity.

      Again, it’s about the OODA loop. They believe they can run a coherent loop. Data is important in all of that, but more importantly — is keeping track of the model, because it’s not just processing data anymore. At some point, it’s also going to be modifying the systems that are then producing that data. Right? It’s a loop. And the most effective companies — it has to be that the data processing is part of both the inference and the execution step. Right?

      And one that was the most shocking to me, honestly, in the last 10 years I’ve been doing this — so many businesses — big businesses, at the heart of a lot of really important parts of the business — the models are very old. They’re very stale. They iterate very slowly. And it’s a massively human-intensive task with VPs and PowerPoints, and everything else to get revs on models. And then you go to the, like, hedge funds, and it’s like, “No, we hire engineers.” They come in, and they code MATLAB, and they’re trading $100 grand the first week. Right? That’s different. That’s a very different view of the OODA loop.

      And, you know, I think in our Twitter exchange, this is where I said — all companies are gonna have to look like hedge funds. Because in a world where you can have essentially unbounded observational capabilities — you can be a logistic startup, and you can basically get data as good as FedEx or anybody else doing logistics. You could be — you can do whatever. There’s a great leveling field with regard to the sensory capabilities. There’s a great leveler with regard to cloud computing capabilities. You don’t need to go hire 100 sysadmins just to go and rack a bunch of servers. You can just turn on some things.

      So with that being said, you can now have extremely low footprint, fast-moving companies that are just there to run the OODA loop, and to have extremely explicit intentional sense-making around the modeling. And for them, data, then — it’s sort of like the difference between a fish — the way a fish sees water, versus somebody holding water to ladle. Right? You don’t even think about the data because you’re just swimming in it. Right? Obviously, you understand data.

      Martin: Yeah. So this is like the silly VC observation. The silly VC observation is if you look at a software company that doesn’t have to deal with the complexity of data, they tend to have relatively high margins, say 70% to 80%. And the reason is, is because they’re building skyscrapers, and then they sell those skyscrapers, and the team needed to build a skyscraper is relatively fixed — and then you can sell as many of those as you want. That’s kind of the software model.

      When we look at companies that are reining in the complexity of data, and that’s how they extract value, the more people you put to rein in that data, the better your results are. And so now your incentive [is] to, like, have more and more people try and work on that data over time. So I think the structure of a hedge fund is — we hire more people to work on the data, we can potentially get more money. Just because they’re actually reining the complexity of that data. But in the software world, all of that complexity is basically going into the margins — yet, depending on who the buyer is, you can’t increase the top line in the same way. 

      So let’s say I’m gonna sell five copies of my software, right? Now, if I sell five copies of my software, people are buying the software. They’re not buying the results of the data. Like, maybe they’ll like my software better because it’s more accurate or less accurate, but the number of people working on the data doesn’t directly drive the amount of software that gets built. And so now you have this existential margin issue, which is — you want to increase the number of people working on the data. Labeling it, cleaning it — because you can always get some improvement.

      Peter: Right. Here’s the question. If we think about — in the software space, you have software vendors and buyers. And the theory of a software vendor, again — going back on our history, there used to just be computer companies. And then Bill Gates was like, “Hey, stop pirating my crap. Pay for my software, because software is a thing. It’s not just your long-haired hippies copying each other’s Unix code. Like software is a thing, right? You need to pay me for it.” Letter to Hobbyists 1970, whatever it was, or something like that. But he did that at the beginning of the PC era. And the PC era basically said, “Well, here’s a set of standards.” Here’s x86. The x86 ISA. Here’s EISA, and BUS, and your peripherals, and networking, and all this other crap. And so you have a set of standards that in the space — oh, actually this recent blog post that I think you — I don’t know if you wrote, but you promoted. The narrow waist of TCP/IP and the…

      Martin: Oh, yeah, that was me, me, and Ali. That’s an old networking guys look at crypto.

      Peter: The point is, you know, a lot of these things rhyme with each other. When you have standards, what they do is they reduce the cost of innovation, and they increase the innovation surface. The PC era was such a gigantic — it’s such a gigantic leveler, that allowed the era of software to thrive. But again, Moses didn’t have a third tablet that said, “There must be software-hardware divided.” And that software must always have these kinds of margins. We’re now entering into an era where people are considering the entire stack of what an information system is. And so, when you look at that, there’s no reason at all, why — if I’m an end-user, customer, buyer — why should X percentage of my alpha, or my margin, or surplus, if you wanna talk about capital and all that stuff — why should this percentage of my surplus go to all accrue — broadly across all these companies — broadly accrue into just one software vendor? Because if I insource it in-house — the technology — and I have the FTEs, all of the residual value stays within the boundaries of my firm.

      And this is what a hedge fund does. In fact, when I go and try to sell to hedge funds, they don’t generally buy software. They use our open source. They like to get consulting services and ask questions. They’re very high-end users of our open source stuff. But they basically say, “Why should I share anything?” Like, they’ll buy a database, they’ll buy some things that they perceive to be truly infrastructure and truly commodity. Anything above that, if there’s a chance of it contributing deeply in a generative way — not a decomposable way, but in a generative way to their alpha — they’re gonna keep it in-house. It’s proprietary.

      I was at a dinner with a CTO of a hedge fund. And he’s like, “Tell me why I should care about open-source.” I’m like, because they had [an] internal, like, crappy version of pandas — and I was trying to give him the story of like, “Look, if you just use pandas, you would basically leverage all of the — you basically have cost amortization of innovation for you,” right, “and it’s not differentiating value for you to have your own little tabular data structure.” People think that open-source is winning, or has won. I think the fact that open source is commoditizing all this stuff means that software itself — the value chain is collapsing. And so, right now, open-source is a movement. I think, unfortunately, it’s confused. There’s sort of this Stallmanesque religious aspect to it almost. And then there’s something deeply beautiful about crowdsource innovation, and legit community collaborative innovation, that’s really important. And we’re almost losing that because everyone’s like, “Oh, but open-source has won now.”

      I think that’s a mystery of the situation. And it’s a thing I keep tweeting about, because I’m saddened by the loss of that thread of the principle. Why do we do open-source? Why do we do crowdsource innovation? So anyway, it’s that conversation. I think software companies do look different because they have thrived in an era of relative — the substrate they’ve sat on is pretty flat. And now we’re entering a space where performance matters a great deal, where the information systems are integrated again. Software is only one component of a whole integrated information system. And because of that, now it’s no longer, like — I can sell just one piece of software across 1000 companies and just harvest all of this margin.

      Companies built on AI/ML workflows

      Martin: So here’s my mental model on these things. Let’s imagine that you have two companies, Company A and Company B. So Company A, they’re building a system, and all the properties of that system are gonna be defined as software. And so they’ve got a roadmap, and then they build the software over a period of time. That’s Company A. Let’s say Company B — let’s say, actually, they’re gonna use just all off the shelf, kind of, AI/ML workflow, but they’re not actually really writing software. It’s all about getting the models to be predictive. And so the entire company is around cleaning data, labeling data, training the models. Right? They’re very, very different, because the complexity of the second one is just far, far greater. And I would say, defensibility of the second one is far, far greater just because of the nature of data. And so it feels to me there’s almost like an emergence of a new type of company.

      Peter: Absolutely. Yeah.

      Martin: Where the organization, the margins, the go to market — everything is being dictated by the fact that they’re processing data, rather than writing software primarily. I think we’re all still trying to understand what that second class of company looks like.

      Peter: Yeah. One of my pitches is that by harnessing the power of open-source to commoditize, to do the disruption on a lot of classical data processing systems, we would basically be one of the last great software companies, and be one of the first great AI companies. The margin doesn’t come from how well you do the software bit. And so, I think that’s the big news. I mean, maybe I have a bit of a controversial view on this. But I think that the era of software being the dominant part of the stack — I know, you know, Marc Andreessen likes to say, “Software is eating the world.” It is eating the world. But it’s a ruminant at this point, right? It’s not the most efficient digester of the value.

      And so, look, you benefit from chlorophyll, even though you’re not a plant — you just eat a lot of plants. <laughter> But I think in the era of — I mean, if we’re gonna, kind of, to go to the — complex systems thinking, right? In the era of data abundance, the people who can build models, refined models, and execute on them, fastest are the ones that are going to win. They’re the chaos agents in the ecosystem. So, look, we still live in a world of plants. But there’s a beautiful infographic I saw the other day, which is how much biomass is on the earth. Most of it is plants. And then you got, like, this little bit is animals. And there’s a little bit there’s, like — this little bit is mammals, and there’s, like, this little bit is humans. I think that in the world order to come, there’s still gonna be, of course, hardware and software companies, so on and so forth. But I think the margins where you really wanna look for the growth is gonna be those people who are moving like animals, and not just claiming a spot. “I’m gonna go here, grow my leaves.” You can still catch some sunlight, but your optionality — I mean, you know, business is war, your optionality is reduced. And the companies that can move fastest among these different places, those are the animals, and that’s going to be running faster OODA loops.

      Martin: I would love to talk about how this impacts the actual business. I’m not sure there’s a huge change on go-to-market, except for the fact that there’s two types of these kinds of AI/ML companies. There’s the infrastructure companies, which basically build the tools to use AI/ML. And that standard — that looks like a standard software infrastructure company. Like, it’d be, like, a data company or something like that depending on your point. And then there’s those that use data science AI/ML to tackle problems in the real world. And in those, it’s kind of interesting, because you end up not building a software company, but more of a farming company or an agricultural company. And so, you’re not selling to core IT right? So they just tend to look very different than typical software problems because they’re selling to a different constituency.

      Peter: They’re not software problems. The software is a means to an end, not the end unto itself.

      Martin: And this is particularly germane to AI/ML, because it allows us to solve problems that typically software hasn’t been good at solving in the past. Like, it allows us to solve vision problems better than we’ve been able to do it before. Audio processing problems better than we’ve been doing it before. It’s kind of like the best way to interoperate with the physical world. And so now we’re off, like, building these companies that solve these kinds of real-world problems. And you just have different looking companies to do that because, again, you’re selling to the person that inspects the HVAC system. You’re selling to the person that is the farmer. You’re selling to the person that does manage the forest.

      I think one thing for the very high level — and, like, anybody creating a company in this space needs to think through is the following, which is — if you’re building just the infrastructure, just the tooling, and the nuts and bolts, you look like a software company, and somebody else deals with the actual AI/ML application. And that’s fine. But let’s say that you yourself are ingesting the data, cleaning the data, labeling the data — there’s a lot of variable costs to do that. Like, every customer may have a new data set. And what happens is this impacts the margins of your business, like, it looks like you have lower margins, because, for every customer, you’ve got all of this work to do. And so I think you’d need to make a decision early on whether — do you want to be the one that’s doing that work, because that’s something you can actually offload to the customer.

      So let’s say you go to a new customer and say, “Listen, we’re gonna take all of your data, we’re gonna clean your data, we’re gonna create your models, and we’re gonna solve your problems.” And in that case, you internalize all of that. And as far as your organization, you need to know that this is basically a services arm. Another option is you can say, “Customer, we’re gonna give you all these tools, but you’re gonna have to bring in your own data, you’re gonna have to hire people to label it, you’re gonna have to learn to tune your models. And we’ll help you with all of that, but you’re the one that’s gonna go ahead and sink that cost.” And so you have to think very deeply of how you structure your company relative to the variable headcount — like, the headcount that has to grow per customer, because that seems to be the big difference that we see for these AI/ML companies, and the typical software company.

      Peter: Yeah. I think it’s hard to do one of these companies right now because we are in a transitional time. A lot of the customers don’t even know what they’re asking for, and they’re kind of looking for that help. And even now, people recognize it as a growth area, and where the future’s headed, so they wanna spend some money on it. But, absolutely right, the amount of work you have to do per customer starts looking a lot like a services play. And there’s a reason why a lot of companies, when you really look inside the skeleton— like, why I think I called it the skeleton buried in ARR. You see a lot…

      Martin: <laughter> Totally.

      Advice for companies in the AI/ML space

      Peter: Eric von Hippel has a great book around democratizing innovation. And he says, “Even when we have a space in which a product is possible, products usually only cover 60% to 70% of the end-user need. The end-user still has to do.” And he’s not talking about software. He’s talking about people like, you know, welding things onto the side of their tractor. He’s talking about, in general, the customer has this thing they need to do. When it comes to the AI/ML application areas, it’s a lot more than just 30%, and it has to be customized per customer site. 

      So I think for businesses right now, in this transition, it’s super hard not to end up looking — if you’re doing a good job for your customers, it’s hard not to look like you’re doing a services play. Now, that being said, there are, I think, viable strategies through this. Which is that you can specialize in an area and domain and say, “Look, we’re gonna come in and work on your data set. But we have our own reference model we’ve built.”

      Martin: That’s exactly right. That’s exactly right.

      Peter: And now we can benchmark you against that. We can bring some of our own magic juice into this. So now the thing that is generalizable across or product-izable across a thing — maybe it’s only for that sector, but the thing that’s generalizable is not just the software, it’s actually more defensible than the software.

      Martin: I just wanna very quickly put a fine point on this. There’s two things that you brought up that are very important to realize. The first one is, we are in a transition. So customers don’t even know what it means to, like, label data and clean data. Maybe in five years, you can go to a customer and say, “We’ve got all the tooling for you, but you’re responsible for managing the data,” and therefore, you offload the cost. It’s just today. You just don’t have enough education in the market to do that. They don’t have data scientists, etc., etc. And so I think in order to get the market into that transition, the startups have to do that. Like, you have to build out that basically — services arm. The second point you made is actually, I think, the critical one is — there actually is some commonality in verticals. And so you can reduce that margin by sharing as much as possible, but it does require customers to share data, or at least share models. And that’s sometimes a tough conversation with the customers.

      Peter: Well, it’s not just sharing models. I mean, there are deeper and interesting, more leveraged plays to be made. For instance, you go into a sector, and you realize, “Oh, all of these people are doing their own craptacular things. These are their limited budgets, and their data sets are broken this way — but holy crap, there’s this other vendor over here with this data set. I can go and negotiate an exclusivity with that vendor. And now I’m the only one that can bring that kind of model lift into this particular sector. So there’s a lot of that 1800s-style, like, homesteading to be done in this space. So I think it’s more than just the “Let me average <inaudible> Central Limit Theorem everybody in this industry.” There’s some really cool things to be done.

      Martin: So the first thing companies need to figure out is what type of a company they are. Many are very confused about that. You need to know are — you a software company and you’re building tooling, or are you a company where the majority of the complexity of the company is around data. And by the way, many companies started as software companies and end up as data companies, and then they’ve structured things incorrectly. So let’s say that you’ve come to the answer to that, and you’ve figured out you’re a data company. Once that happens, you need to understand that often companies that are extracting value from data — there’s a lot of complexity per customer in order to do that. And you need to structure your company the correct way, which is like — just realize it may be hard to scale, just realize you’re gonna have different processes around the actual data. Or come up with a strategy to offload that to the customer.

      Now, the reality is, because the market is so immature, it’s unlikely the customer is gonna be able to do a lot of that, but it’s something that you can, over time, train the market to do and do that transition. But I think this is the big sticking point with many <inaudible>. They think they’re software companies. They end up being data companies. They didn’t build the organizations to deal with that intra-complexity. It’s coming down in the margins. Everybody is kind of confused. And so I think just a little bit of self-awareness and a little bit of planning go a really long way in this space.

      Peter: But it requires a very different — many West Coast firms have the thesis that to do a really great tech startup, you need at least a tech founder somewhere in there, because they kind of see where things are going. For a really good AI startup, you need to have machine learning people at that leadership level because they know what it means. They know why a single data set can be a billion dollars, or swing a billion-dollar deal. The difference between a software engineer and, like, a data scientist is that — software, you generally know what the inputs are, or the types of inputs, and your goal is to construct a system that, given these inputs, produces these sets of outputs. So you have very nice, clean definitions around correctness, for the most part.

      With data science, there’s unfortunately not that. You can have a piece of code, and for some sets of values, it’s correct. Other sets of values, it still produces a result, but those results are wrong. And a function’s correctness is dependent on values. This is the key thing that differentiates all of data science — from machine learning — from classical software engineering. Classical software engineering, it’s like, we’ve got our test data set, we’ve got our prod data set. It works in test, it’s gonna work in prod, right? That’s not how data science machine learning works at all. In data science, machine learning, the correctness of a function is value-dependent, and also performance-dependent — and the performance also value-dependent.

      So now you have this intertwined synthesis of a data, and a modeling, and a computation problem that cannot be decomposed into orthogonal vectors, right? That’s the difficulty of this. What I think is that in 5, 10 years time, every company that is actually still in existence and doing well has to, essentially, have synthesized and brought a synthesis in of their data capacity, their data modeling capacity, the model build, and computation — the hardest thing is appropriate computation — and economical fashion to suit their needs.

      So the word I like to use for this is cybernetics. I mean, we are right now in between the software era and the cybernetic era, and I think we will get to a cybernetic future. And cybernetic, by the way — you know, it comes from the same word as Kubernetes, right? It means governor. It means a theory of action and control. So businesses have to see computation really moving its way up. Data modeling process has to move all the way up to the very tippy-top of the business. That synthesis will happen, it will have to happen. And that’s what the selection pressure is in the business world. I don’t know exactly the path we’ll take to get there. In the transitional time, businesses who want to basically get in ahead of the curve, they’ve got to have very clear thinking at the leadership level. And they must have a very clear understanding with their investors about what they’re gonna look like as they chase the marlin, because it’s gonna take a little while.

      So I think that’s the trick right now, is that you’ve got to find founding teams or leadership teams that have a solid understanding of software — of what software is and isn’t, of where the value is in the software activity. And of where the value is in the data and data modeling activities. In a time of fog, you’ve got to have very, very clear-headed thinking about that sort of thing. But ultimately that synthesis must be what comes.

      Martin: Thank you.

      Peter: Thank you so much.

      • Peter Wang

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      Working, Making, and Creating in Public and Private

      Nadia Eghbal and Sonal Chokshi

      We’re living in an unprecedented era of online collaboration, coordination, and creation. All kinds of people are coming together — whether in an open source project or company, an R&D initiative, a department in a company, a club or special interest group, even a group of friends and family — around some shared interest or activity. But the word “members” is faceless, and doesn’t help us really understand, support (and better design for) these communities.

      So in this special book launch episode of the a16z Podcast, Nadia Eghbal — author of the new book Working in Public: The Making and Maintenance of Open Source Software published by Stripe Press — shares with a16z editor in chief Sonal Chokshi the latest research and insights from years of studying the health of open source communities (for Ford Foundation), working in developer experience (at GitHub), researching the economics and production of software (at Protocol Labs), and now focusing on writer experience at Substack.

      Eghbal offers a new taxonomy of communities — including newer phenomena such as “stadiums” of open source developers, other creators, and really, influencers — who are performing their work in massive spaces where the work is public (and not necessarily participatory). So what lessons of open source communities do and don’t apply to the passion economy and creator communities? How does the evolution of online communities — really, social networks — shift the focus to reputation and status as a service? And what if working in public is also about sharing in private, given the “dark forest theory of the internet”, the growing desire for more “high-shared context” groups and spaces (including even podcasts and newsletters)? All this and more in this episode.

      Show Notes

      • What “open source” means [1:56], types of communities [4:17], and how they control growth [7:19]
      • The modular nature of open source platforms [10:16] and the ideological framework driving open source software [12:48]
      • Further discussion of managing growth and the creator’s time [15:12]
      • Open source contributors who create their own brands [20:10]
      • Discussion of platforms that are abandoned [22:34]
      • Subscription models and building an audience [26:51]
      • Platforms that are deliberately outside the mainstream [31:27] and their relation to newsletters, email lists, other semi-private spaces [34:43]
      • Crisis of the commons and how it relates to online platforms [36:37]
      • Guidelines for community managers, platforms, and communication tools [42:15]

      Transcript

      Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and I’m super excited to do one of our special book launch episodes, for the new book coming out just this week — “Working in Public: The Making and Maintenance of Open Source Software” by Nadia Eghbal, and published by Stripe Press. The topic actually applies to all kinds of communities and groups coming together, whether it’s an open source project, an R&D initiative of a department in a company, a club or a special interest group — even a group of friends and family, because it’s all about how people come together to coordinate and collaborate around some shared interest or activity — whether participatory or not, whether code or content. 

      And so, one theme we also pulled the threads on in this episode is about how the learnings of open source communities do and don’t apply to the passion economy and creator communities as well. Nadia has long been immersed in studying the health of communities, including getting funding from the Ford Foundation to study open source, then worked at GitHub in developer experience, then did research at Protocol Labs, and is now focused on writer experience at Substack.

      For longtime listeners of the “a16z Podcast,” I’ve actually had her on the show years ago — along with Mikeal Rogers of Protocol Labs, then of the Node.js Foundation — where we talked about the changing culture of open source. You can find that episode on our site. But in this wide-ranging hallway style episode, Nadia and I cover everything from types of communities, social networks, and the evolution of being online. And, ironically, while the book is called “Working in Public,” we also talk about the emergence of private spaces, as well as the tragedy of big public commons — and how to counter the tragedy of commons, which is why I believe everyone should read this book. Because there’s a dearth of literature out there for the era of unprecedented online collaboration, creation, and consumption that we’re in. 

      We end with some quick practical advice for community managers, platforms, and leaders, but we begin by quickly defining open source in this context, with a really useful taxonomy for categorizing communities.

      Defining “open source”

      Nadia: You know, early on, I was just like, “Oh, I really hate this term.” And I just wish we could have gone with something else, like public software or whatever.

      Sonal: Ooh, I love that.

      Nadia: I love it, too. Unfortunately, it’s pretty hard to change terms that everyone else agrees on.

      Sonal: Yes, I know this firsthand.

      Nadia: Yeah, I mean, I personally find the term kind of intimidating. And I don’t know, it doesn’t sound exciting when I say the term open source. But it really does just refer to the distribution side of code. The existence of open source licenses made it very easy for anyone to use and modify and republish someone else’s code, then put it in their own software. But it doesn’t really say a whole lot about how open source is actually produced. And so I make this analogy in the book — which is actually an analogy I borrowed from my friend Devon.

      Sonal: Devon Zuegel — she’s hosted a couple of podcasts for me. I love her.

      Nadia: Yes. And she says something like, “The term open source doesn’t mean anything, any more than the term company does.” It’s like, yes, we kind of get what a company is. But there are so many different kinds of business models for different types of companies — and so, similar is open source. Saying something is open source tells you a little bit about how the code might be used, but doesn’t really say anything about how they’re actually being made. Someone has to continue taking care of it.

      Sonal: One of my favorite parts of the book is how you actually outline different types of communities. You call it classifying project types, but it’s really, to me, how people organize and, like — essentially social networks, really. So why don’t you break down that taxonomy. And, by the way, the reason I’m asking is, because when I think of the arc and history of open source, the concept that comes to mind for everybody is that classic book by Eric Raymond, which is “The Cathedral and the Bazaar,” and I think that framing has too long framed our discussion of open source, and frankly any online community.

      Nadia: Yeah, everyone sort of has this maybe, like, general understanding of what community is — like, there’s a bunch of, like, members, and they’re kind of organized around some common interests, or reason for spending time together. In that highest level definition of a community, there’s an underlying assumption that, like, all members are sort of similar. And just the term “members” sort of, like, washes over the underlying dynamics between those different members.

      And so, what I started by doing was saying, “Okay, there is a difference between — at minimum, in open source — people who are contributing to open source, and people who are using open source.” So I try to sort of separate out users and contributors and say, okay — in some open source projects — or, as you said, really communities in general — some communities have high contributor growth, and some communities have high user growth. And then there’s sort of, like, different permutations of that.

      Types of open source communities

      Sonal: It’s like federations, clubs, stadiums — and I forgot the fourth — toys. And tell me what those are. So I think it’s really useful to start with your taxonomy of federations and onward.

      Nadia: So federations are like the really big open source projects we might be thinking of — like Linux or Wikipedia — where you have a lot of people who are contributing to the project, and you have a lot of people that are using that project. But there are enough people that are working on the creation of that project that, like, it does form its own sort of contributor community. By contrast, clubs have a lot of people who are participating in its creation, but they don’t have as many people that are using it. And so that product that’s kind of focused on a niche interest — the example I like to give is Astro Pi, which is a Python library for astrophysicists.

      Sonal: Right. It’s high contributor, because they’re incredibly interested in that, but very low user growth — because how many people in the world are really interested in that intersection?

      Nadia: Exactly. And toys I sort of mentioned in passing, and they’re probably the least interesting thing to talk about. That’s where you have both low user and low contributor growth. So that might just be, like, a personal project that I’m tinkering around, no one else is really looking at it. They’re sort of waiting in the wings before they become one of the other types of communities.

      And then the fourth model is stadiums. And this is the one I think is most interesting and most overlooked, because it’s kind of a newer phenomenon. And so this is a situation where you have one or maybe a couple of contributors, and then you’re making something for, like, a very large audience of users. And so you can imagine someone’s standing in the middle of a stadium — there’s this imbalance where, in this case, the developer is feeling a lot of inbound requests, a lot of comments, issues, pull requests — just a lot of needs from their users.

      But there aren’t that many people who are actually able to help. Contrast it to a federation, where you imagine something like Linux — is extremely widely used, but there’s also a very mature and well developed ecosystem of contributors to support it. But a big part of this book is taking the time to stop and look a little bit more deeply at — what is that giant audience of faceless users, and are there interesting dynamics happening there that actually make this look more like a community. Where, like, a stadium is actually a legitimate type of community that stands alongside the clubs and the federations. We just haven’t really taken the time to understand it before.

      Sonal: I love that, Nadia. What I found fascinating about stadiums is — you’re essentially describing — and I think about this as someone who cares about content and social, is a rise of an influencer. This is no different than influencer economies, in many ways, where you have sort of, like, a star contributor and then, like, a bunch of people kind of in this stadium — literally, in your analogy, watching them. And you even say that it’s this shift — and I think you’re quoting someone — to facing the stage, versus facing each other. So when you have this person who’s on the stage, and they’re, like, the primary contributor — and let’s just say creator, because we’re essentially also talking about creator economies here. <Yes.> You made this distinction that they may be intertwined and influenced by their community around them, but they’re not actually “doing peer production” in the classic collaborative way of the first eras of open source. So can you explain that a little bit, and tell me a little bit more about why that’s happening?

      Nadia: Definitely, it does have this broader application to what’s happening to individual creators on all these different social platforms today. Most open source projects, we can probably say, used to be clubs — where just, like, not a lot of people were using open source in its earliest days. You kind of had this group of weird developers who just loved using it and maintaining it for their own purpose. And then, eventually, we kind of hit this point where open source became so popular that tons of people were, kind of, discovering all these projects and using the code. And I would attribute that in large part to the creation of a platform — GitHub — which kind of united all these projects together and made them discoverable in one place, with a much more standard way of contributing and discovering — and just, like, thinking about what is open source. 

      For a lot of people, GitHub is basically synonymous with open source. Another useful parallel trend here is just that — open source projects started becoming a lot smaller, due to just platform effects of different languages having these package managers that made it really easy to find and discover and use lots of different libraries. And so, now these projects are smaller. They have one developer at the helm, but they have 10x, 100x, 1,000x more users that are coming in.

      And so, suddenly you go from having these clubs where everyone kind of, like, knows each other — if you’re using it, you’re kind of expected that you will be contributing back if you need something, instead of asking someone to do it for you. Suddenly you have, like, all these outsiders that are, like, flooding into a project and using it. At the same time, you also have some portion of those users who are now coming into the project and asking for things. And they don’t have the same background that the core developers or creators or maintainers do. They’re just sort of, like, asking for things and leaving. 

      A useful analogy here might be thinking about a small town that was largely undiscovered, was not connected to a highway — and then once it became part of a highway system, then you suddenly have all these tourists who are now stopping by some cute little town. And suddenly, it changes the nature of the entire town. Because, I mean, in some tourist towns, you can have — more than half the population is actually tourists, and not even local residents. It’s either, “I’m just gonna completely close off and do my own thing, or have to, like, welcome everyone.” And those are kind of, like, the two extremes that I often hear about when they’re trying to, like, think about, “How do I manage this volume?”

      And so, what I’m sort of trying to suggest is there’s something in the middle there — where it’s okay to make things and share them in public. But it doesn’t mean that everyone has to participate. And that’s a theme that I really tried to push on in this book — is that something being public does not mean that it has to be participatory.

      Sonal: You actually shared a great analogy in your book, where it’s like the Twitter user — like, an early Twitter user, before they become kind of famous or big. They’re very good about, like, responding to people. They’re building their community. It’s very, like, peer-to-peer. And then there’s a point where some of them become even more influential. And they’re so overwhelmed by mentions and replies and questions that they can’t even remotely respond to any of it — let alone little of it. So I thought that was a very useful analogy for thinking about that. Because one can also evolve over time. 

      Modularity of open source platforms

      So you mentioned that there’s this, kind of, increasing packages — where people can kind of take things and combine them. And this really stood out to me, because one of my absolute favorite themes when I think of, sort of, meta themes for innovation, and how people change the world, and how people change things — is modularity. And I have this, kind of, joke of, like, “Modularize all the things.” Let’s talk about why that modularity matters. And the example that we both know is modularity in the form of, like, the node package manager. Our mutual friend, Mikeal Rogers, ran much of the work in the Node.js community. Let’s talk about how that shift has mattered.

      Nadia: So, on the one end you have very monolithic software, where if you change one thing, it has a lot of patience for changing other things. Software products that look like this tend to be a lot more thoughtful and deliberate and slow about what they actually want to accept as a contribution, or changes that they want to make. Because the whole thing is tightly coupled — but also, sort of, brittle in that way. And so it has just a very different implication for, like, how many contributions we actually accept — how much can we actually change things?

      What happened when open source became a little bit more modularized — which is probably best exemplified, as he said, in the story of npm and JavaScript — is that now instead of having this tightly coupled code — you can imagine, like, a tower made out of Lego blocks. Where you can remove one of those blocks, and, like, the rest of the tower still stands. So, it completely just, sort of, changes how we think about a single piece of software. And that, like, instead of having to think about the major implications of changes between the different parts of the code, you can actually say, “Hey, I’m gonna grab, like, lots of different components from different types of developers. I want this person’s library and this other person’s library” — and just, like, fine-tune it to look exactly how you want. And as a result, it enabled a lot of new creation in open source.

      Sonal: In the crypto world, the community and the team here loves to talk about the composability of open source projects. Composability being the idea that you can take these building blocks — you mentioned Legos — and that’s really important, because people are combining, remixing, and reusing. And it’s kind of a buzzword, but I use it on the podcast — I’m gonna stop being ashamed of it, which is — combinatorial innovation. And it’s very “primordial soup.” Like, you get all these ingredients, and then it leads to this combining and recombining and evolution, and the Cambrian explosion. I’m just throwing [out] a ton of buzzwords there. So, that’s why the modularity matters. So now, can we talk for a minute about what it means from a project point of view? When open source goes from big to this small kind of — collectives of people that may come together, what are the implications of that?

      Nadia: So, if you talk to free software activists from, let’s say, the ’80s or the late ’90s…

      Sonal: I used to edit one of them — Richard Stallman. And he would call that “Libre.”

      Nadia: Yeah, people that are really focused on the sort of, like, ideological implications of open source or free software — if you talk to them, you’ll find that — or I, at least, found — that a lot of them are really concerned about the liberation and protection and longevity of the code itself. Like, freedom is not referring to any freedom of developers. It’s referring to freedom of the code. <Right.> But if you kind of come down to, like, a world where things are a lot more modularized, suddenly the focus shifts from the code to the people who are behind it — because now every piece of code is much smaller and more trivial. 

      There are very well-known developers, especially in the world of JavaScript, where that really encourages a lot of this, sort of, style of development. There are very well-known developers who make hundreds or thousands of popular npm modules — which are each their own separate project, but they’re — each one’s very small. And so, suddenly it kind of becomes more about the person behind it. A useful parallel here might be thinking about the impact of tweets versus blogs, where a blog post is this, like, lengthier thing, and a blog post kind of stands alone as this beautiful piece of literature or whatever.

      But then, like, if you’re really into using Twitter, like, you might tweet like 100 things in a day — and one tweet might go viral, but, like, you have so many more that come up, right? And so, it kind of just becomes, like, about the person tweeting. It’s not about, like, “Oh, he wrote that amazing tweet six years ago that I often revisit.” Like, that’s not really what it’s all about. And so, I think — to sort of summarize this — I think this shift towards modularization also helps drive why we’re seeing more interest in reputation-based and status-based economies. <Yes.> Because it just, like, wasn’t the factor before. It was all about the code. Now, it’s all about the people.

      Sonal: Mikeal Rogers and I actually wrote a piece about this when I was at Wired. It was called “The GitHub Revolution.” And this was like in early 2013. And, basically, the fundamental point is that GitHub inverted the model from project to person, and then identity came [into] the picture. But to your point, when you have these modularized packages, and individuals who are very tied to that, it does become about the person. But now, on the social side, if reputation and the person is at the center — not just the code — what does that mean for how these groups organize? And what does it mean for how they manage and how they collaborate?

      Managing growth and the creator’s time

      Nadia: Yes, there are absolutely different implications for how these different types of communities can and should think about organizing, and how they think about growing and maintaining over time. The currency that I’ve settled on was focusing on a producer’s attention as a limited resource. So we all talk about the attention economy, but the attention economy tends to refer to a consumer’s limited attention. But we don’t often talk about a producer’s limited attention. So, like, a creator only has a finite amount of time as well. If we’re thinking about creators and not these, like, big distributed communities now, the creator is kind of, like, on their own, and their attention is not gonna scale by themselves. 

      The first line that I would draw is between clubs and federations, which are dealing with an abundance of attention, because they can be high contributor growth. And then stadiums that are dealing with dearth of attention, I guess you could say, because their contributor size is not growing significantly, but their number of users is growing.

      Sonal: Right. And just again to emphasize, you’re talking about the attention of the contributor and the creator?

      Nadia: Yes. And the ones that are probably most interesting to talk about today are the difference between clubs — which have high contributor growth and low user growth — and stadiums, which have high user growth and low contributor growth. And so, one of the things that previous online community literature focuses a lot on — and especially also in open source — are governance processes. And governance is probably more useful and important to talk about in the context of larger contributor communities, because these are coordination problems, right? Like, you have multiple members with a stake in the community who are all coming in with their own interests. And you’re looking to figure out, like, how do we all best work together?

      On the creator’s side, there’s probably another version of these processes that need to be developed for stadiums that’s not really about governance, in the same way, because you usually only have one or a couple people that are at the helm. It’s more about the relationship between that creator and their audience and, like, “How do I interface with my audience? How do I make them feel heard? How do I utilize people that might be willing to help or pitch in?” So there’s a lot of just, like, different kinds of strategies they can think about around, like, how do I — given my limited amount of tension, like, how do we make sure that stuff continues to get done.

      Sonal: Right. To pull on a couple of other threads there — does this mean that these relationships even have to be persistent? I want to hear your thoughts on that, because we talk about these very stable federations that have been around for decades. But one thing that I find very appealing — and might be a bug to you, but I think is a feature — is that some of these things seem like they don’t have to be persistent and can maybe be very ephemeral, when you have that kind of small modular setup.

      Nadia: I absolutely think the relationship between creators and their audience becomes a lot more ephemeral. And we should almost be, like, leading into that design, right?

      Sonal: Yes. I really strongly believe this.

      Nadia: Yeah. And so, like, there have been these terms that have existed in open source for a while — the idea of, say, like, a casual contributor — to distinguish between someone who’s kind of dropping in and making one contribution, versus someone who’s a more, like, active or present community writer.

      Sonal: Right. Didn’t we even call them — I think in our last episode — they’re drive-by contributors, right? <laughter>

      Nadia: Yes. Drive-by contributors, casual contributors. And so these are the people that are not coming in with a pro-social attitude. One thing I did find in my research is that folks that come in as these more active contributors making substantial contributions — a lot of them do come in displaying pro-social attitudes from the beginning.

      Sonal: Ah, interesting.

      Nadia: Yeah. So they are coming in saying — they’re looking for a community that they want to be a part of, and they want to help out. So, like, one behavior you might see that’s different about an active contributor versus a casual one is someone coming in and, like, answering someone else’s question, instead of opening an issue saying, “Fix my thing.” It’s, like, two very different kinds of behavior, right? Like, one, you’re trying to help someone else — and, one, you’re asking for help. Like, “I want to get something out of this. I want to get my contribution merged. I have a question that I need answered,” whatever. They’re coming in with some sort of personal interest.

      Sonal: By the way, you also use the word parasocial in your book, which I had to look up because I didn’t even know that was a thing.

      Nadia: I actually think, like, parasocial is a great way to just describe what kinds of community these stadiums essentially are — which, it basically just means, like, one-sided communities. Where, like, one side of the audience has a deeper, more perceived intimate relationship with the creator than the creator does to them.

      Sonal: That’s very similar to podcasting.

      Nadia: It is very similar. If a creator were to treat every single fan that they met or every single person in their audience as someone that they’re gonna develop a deep and long-lasting relationship — like, that’s just exhausting. It’s completely impossible. But if they say, “Okay, like, we are gonna just meet this one time. Like, how can I make sure that this person feels fulfilled, or whatever, and I manage this without giving too much of myself?” And so, yeah, like, these interactions are more ephemeral. And we can, sort of, design around that where, like — “Here are a bunch of, like, self-serve resources.” Or we can encourage users to help each other, instead of always turning to the creator for help. And so all these other, sort of, like, supporting satellite communities can thrive and flourish on their own without needing an involvement from the creator.

      Open source creators and branding

      Sonal: What do you think of something like “The Ringer,” where you have someone like Bill Simmons — the analogy here is, he’s a hotshot coder — but, really, he’s like, a creator. He did “Grantland,” and then he went out on his own and did “The Ringer.” And then within that, he built a constellation of brands underneath his parent brand. It’s both bundling, and also, like — just constellation communities. Do you have thoughts on how that works? And how that might play out in the open source world as well?

      Nadia: Well, I guess there is a version of that that happens in open source, which is — you have this broader language ecosystem. I’ll keep coming back to JavaScript as the best extreme to demonstrate this. And we can drill even further into JavaScript — let’s say like the React ecosystem. And within React, there are a bunch of associated projects that a React developer might use. And so when we think about who is a contributor to that project, like — yes, you could look at who has actually made contributions to some specific subproject. But you could also say, “Well, who’s contributing to, like, React more generally?”

      And so taking, like, webpack — or something that is a subproject that a React developer might expect to use — someone might have never contributed to Webpack before. But if they’re well known as a developer in some other part of the React ecosystem, then they already have a little bit of currency and a little bit of reputation if they were to try to come in and open a pull request, or make a contribution. And so I don’t know exactly what the analogies are between that and sort of, like, subscription bundling, or what that can look like.

      But one thing might just be that when we think about — what would it look like to have more subscription-type support for open source developers — which GitHub Sponsors, Open Collective, there are examples of this already — we might think a little bit more about, well — it’s not just this one project that this developer works on, but they work on this ecosystem more generally. And so, maybe similarly — the way that, like, a writer might have started with, like, one type of newsletter, and then, like, they join forces with another one — and then, suddenly, we’re sort of supporting this entire bundle of people that are working on a similar theme. You can imagine that happening with open source developers, where they’re no longer just tied to, like, one specific project, but it’s like, “I support your development work more generally.”

      One of the more obvious examples, I guess, I could point to is Sindre Sorhus, who has done pretty — like, thousands of mostly npm-related projects. <Right.> But he’s sort of, like, his own mysterious entity. It’s not really about any one specific thing that he does. <Right.> He’s just, like, a very generative person. And he is supported through sponsorships.

      Sonal: I’m gonna ask you a crazy question. This is a thing I’ve been very fascinated by for a long time. I tried pitching this at Wired — by this idea of, like, digital suicide. Taylor Lorenz writes these beautiful pieces about, like, Instagram, and all these various communities online, etc. And I’m also fascinated by this phenomenon of all these, like, teens creating multiple accounts and multiple identities on their Instagrams. And then they also abandon them, which is something I love — this idea of this kind of abandoned wasteland of digital identities and places, because it feels like the real world to me. That there are places that are ghost towns, and places that have been lost in the sands of time, for better and worse. Do you have thoughts on how that may or may not apply to open source, because not only do these things not have to be persistent — they can be ephemeral. Is it okay that they die, or that they even have — up front — a calculated, kind of, end point?

      Nadia: Oh, this is where software gets really interesting — and, I think, different from most other forms of creation. Because if someone creates an Instagram account that gets really, really popular, and everyone’s following it, and then eventually — suddenly — this person goes dark, and we never get another post of them — a lot of people will be sad about it. People might create, like, spin-off accounts, and tribute to that original account, whatever. But, like, the world doesn’t actually, like, break and shut down. 

      If a maintainer has a product that is wildly popular, and they’re just sort of, like, over it, and they disappear — and this does happen often — that code is still — if it’s popular, is being used by a bunch of other people. And, like, code changes over time. It doesn’t need upkeep and maintenance. Intrinsic motivation helps a lot with — on the creation side of things in the very beginning. If something becomes really popular, then you start getting these more extrinsic rewards, like reputational benefits, or status, or whatever. But a lot of stuff is sort of front loaded.

      And so, if you’re talking about maintaining a software project into perpetuity, after a while, you’re already known as the creator of that thing — there’s not really any additional benefit. But you still need to keep maintaining, and, in fact, sometimes those maintenance costs get even higher over time. And so, that’s why I think it helps make the case for — we need to find other reasons to keep people wanting to maintain stuff or make it easy for them to step down. Because intrinsic motivation really only takes you so far. And so, if something happens, where they actually, like, need some changes to be made or need updates to be made to the project, and the maintainer is nowhere in sight — and nobody else has the ability to, like, make commits or contribute to the project, like — this actually can create, like, real problems for software.

      Sonal: You’re basically describing software as more of a living, breathing organism, actually, in that context. <Yes.> You either want to evolve it and keep it going, and generationally it can evolve into something else and have offspring, etc. But it’s a different thing than when you just have, like, an abandoned site or, like, an abandoned farm somewhere.

      Nadia: And this is why, like, from the beginning, the ability to fork code or basically, like, copy the exact repository somewhere else — has become — it was, like, a very important part of it early on to say, like — someone can always take the copy of this code and make their own version of it somewhere else. Unfortunately, this comes back to this dependency issue today, where — yes, in theory, you can fork a project. And in practice, there might be a lot of other things — other software libraries, other pieces of software — that are pointing to that specific project. And so, if you fork it, you now have to somehow convince all those projects to start pointing to your new project. 

      And so, it is this challenge with open source where sometimes, like, a maintainer disappears and is nowhere to be found. But, like, they still need to keep doing things to the code and to the project. Forking is not always an answer — an easy answer to that. It’s still about telling everyone like, “Hey, come over here. Use this.” And that’s actually why I think this concept of maintenance, that is maybe easier to see in software, actually really applies to every creator today. <Yes!> Because a lot of people go through this experience of, you know — you did one thing that might have gotten you this huge, like, seed initial audience or whatever.

      Sonal: It made you internet famous too, for lack of a better phrase. <crosstalk>

      Nadia: You can have that moment, but then you know, you have to continue creating things — otherwise, people are gonna stop paying attention to you. Much like writing code, it’s not enough to just sort of, like, publish it once and be like, “I’m done. Like, I’m never gonna touch this thing again.” If you are trying to build this reputation over time — and some people will say, “I had one viral video on TikTok, and that’s it. Like, I’m never doing anything else again.” But if you’re trying to be, like, a TikTok creator, you’re gonna have to start making more hits over time. And so your reputation is itself this thing that requires maintenance in order to stay relevant.

      Building an audience

      Sonal: It is literally one of my favorite parts of your book, because it reminds me of the theses that we’ve talked about at our firm, too, around the passion economy. Here, the artifact is code, but it can be any activity that’s being coordinated, quite frankly, in your framework of your book — which is why I really believe, again, that this book is applicable to everybody. Open source is almost a misnomer, because people think it means code. And it really means everything. It’s like any kind of creation and consumption, frankly — but what you’re really saying is a maintainer is not just a coder, it’s a creator. And they’re maintaining their content in this world, or whatever they’re creating — which I think is incredibly powerful. 

      And what’s really powerful about that is — then you think about sort of the related business models for that. Like, when I think of the example of what subscription and SaaS, Software as a Service, did for the world of on-prem software and how people used to sell software — and you had, like, the suited person do this big multimillion dollar deal install, never see them again. SaaS changed the game for everybody in companies because you had to consistently earn their dollars every month, but in a way that was a wonderfully sticky, stable relationship too, like you were mutually dependent.

      Nadia: Yes, this is, I think, a giant red arrow pointing at why subscription models are gonna only become more and more interesting in the very near future, because they do take into account this need for, like, ongoing development. There are ongoing costs associated, and you have to earn that, as you said, over time. And so, they’re capable of sort of capturing both your existing value — the value that you have accrued thus far — and also speaking towards the future value that you might create, because when you subscribe to someone or someone’s thing, you’re saying, “Like, I expect there is gonna be more stuff being created in the future.”

      Sonal: And this goes back to the phrase, and something you said in your book, and that’s sort of theme for me — you can be transactional, but be in a very high sustained relationship, because it’s a repeated game — which is what subscription is. I think that’s super fascinating. So what do you think the implications are, then, for people who change clubs. They go to a different — they create a new stadium. Like, how has this new passion economy and model evolved?

      Nadia: I think this kind of comes back to the value of platforms. And they’re a distribution power. So, I mean, in theory, in the past — without having a platform, which is essentially just a stage for creators. That is always gonna exist, is always there for the creator — without that, if you wanted to go off and, like, start something new somewhere else, it’s really, really hard. Because how are you gonna direct anyone to your new thing all the way over there? <Right.> It’s like building a house but not building a road to the house. 

      Whereas, like, platforms have this very important role that they play for creators, where if you want to do something new, you have an audience that you’re building on there that you can use to seed whatever your new ideas. Platforms make it so much easier than you could have in the past. I mean, this is also — so I work at Substack, and this is also why I and everyone that I work with believe really strongly in the power of an email list, because an email list is something that you own. And if you want to do something new with it, if you want to do something totally different, like — you have an audience that is sort of built in and that you can take around with you wherever you want. But even if you don’t have an email list, like, having a Twitter following or having an Instagram following, or whatever, gives you that sort of, like, seed money to do something else.

      Sonal: Basically, you’re saying that you have the distribution because your audience travels with you. And that’s an important currency because you don’t have to start from scratch every time. That does go to your other point as well, that the reputation is the key and the currency there. And that’s where status — and you talk about this in your book, and Eugene Wei’s thesis about “Status as a Service” comes in.

      Nadia: Eugene’s thesis came out, thankfully, while I was writing my book, and it was very helpful, because I was like, “Okay, now I have more vocabulary to explain the things that I mean, that I’ve been struggling with.” I think actually this framing of status economies helps explain some of the shortcomings of GitHub thus far, because there isn’t, sort of, a meaningful way to measure someone’s status — or just have a clear picture of what someone does on the platform, or what kind of developer they are. You can look at any one specific project, and you can see how popular it is, you can see how many stars that it has. But if you go to a developer’s profile, it’s not super clear what they’re known for. You can technically follow a developer on GitHub, but it doesn’t really mean anything — not at all the way that it does on Twitter or something like that.

      And so, I think if you talk to well-known developers, or developers that have these larger followings, they’ll probably tell you that they keep their audience on Twitter or somewhere else. And GitHub serves a little bit more of this utility function, as Eugene said. Where if a platform fails to provide this sort of status benefit, then it basically becomes a utility. They will continue to develop the social and status aspects of their platform. But right now it really is much more of a utility, I think.

      Sonal: So, you mentioned the power of a platform. And you’ve been using this analogy of, like, cities, and highways, and connecting houses, and connecting a small shop or a small village to a highway — and what that does for people. What about the opposite, when people go off the grid, essentially, and go outside our purview into these sort of private, dark social places — whether it’s WhatsApp groups or Telegram groups, or private stadiums, private groups. And you mentioned in the book — and I saw Yancey Strickler’s tweet about this when he did it on Twitter — he’s a former co-founder, CEO of Kickstarter. He draws the analogy of the dark forest. The reason that we can’t communicate with aliens is because the world is so vast, and the only way people can protect themselves is by being in this dark forest, where there are these vast spaces of separation. So you’re not in this vast — you’re not actually in what is commonly referred to as a public commons. You’re actually very isolated.

      Nadia: The only thing I would maybe add there to the dark forest concept or metaphor, is this idea of hostility — that we are all, actually, surrounded. There are all these other people out there. If we’re sitting here wondering, “Where are all the aliens?” They’re there, but the theory being that we’re all trying to stay out of each other’s way and not be detected because…

      Sonal: Destruction will be the result.

      Nadia: Yes, it’s not a good thing to meet anyone else as curious as we are.

      Sonal: The dark forest comes from the idea of the Fermi paradox, I believe — and I’m a big fan of “The Three-Body Problem” trilogy. They have the wall facer — he’s the one who figures this out. So I thought that was a super interesting analogy. Tell me a little bit more about your thoughts about the dark force theory of the internet, and how that applies here. What happens when people go off platform?

      Nadia: So Yancey Strickler’s comment about this. And I think basically a lot of people are observing that, okay — we started with these really big social platforms that have grown to become really big — so, the Facebooks and the Twitters and the Instagrams and YouTubes of the world. These are sort of, like, the biggest stages possible. And so, the analogy to what’s happening on the very public web right now is that everyone’s still talking, it’s just sort of, like, we’re kind of moving to these little corners, without fear of being attacked or jumped on somewhere where all context has otherwise collapsed.

      Sonal: I’ll say one more thing, because I’m a big fan of the work of the sociologist Ronald Burt — and he talked about this concept of structural holes, where you can have, like, clusters of activity and networks. And I came across this because when I used to work at Xerox PARC, we used to talk a lot about the innovation that would happen when different fields would collide. And it’s because you have these containers — these clubs, these stadiums — of people who have strong ties, but then these really interesting things can happen with what are the weak ties, and then the structural holes in the network. So, if you map these out as, like, a universe of clusters, imagine what’s possible when you can actually bridge some of those structural holes across communities. <Mmm.>  Like, your book made me think about that, actually. I wonder if that’s where the future is going. Is a bundle maybe that? Who knows what’s happening there? I mean, we’re only at the beginning of it.

      Nadia: Yeah, I mentioned this quote in the book, but Kevin Systrom said in an interview in 2018, I believe, that social media is in this pre-Newtonian age where we know that it works, but we don’t know how it works.

      Sonal: Ah, I love that.

      Nadia: I just think that’s really perfect.

      Sonal: It is. It’s perfect for the time we’re in. And it’s perfect for why your book is so relevant.

      Nadia: This is where I think the model of clubs versus stadiums becomes really useful. For a long time, everyone was really focused on, like, the highly public aspects of the social web. But people are now starting to look at the semi-private web and these quieter spaces. The biggest parallel trends that I’m seeing right now — like, one is seeing this formation of these creator-oriented communities that look like stadiums on the big public stage — in, say, like Twitter or whatever. But then you see this other emergence of, like, group chats. And group chats have become this really — I mean, [they] have always existed and [have] kind of become a much bigger thing in recent years, <Yeah.> partly because people are looking for a relief from this high, heavy public space. And those map really well to clubs, where you aren’t trying to add a lot of users to your messenger app. You’re trying to just keep it to, like, six of your closest friends. In most cases, we’d say that you’re, like, actively suppressing user growth. But contributor growth is high, where you’re totally down to chat with your friends and that little group.

      So, those map really well to the clubs that I sort of identified here. Whereas stadiums apply to both these, like — creator communities are happening in very public platforms but I think can also help us understand why things like podcasts and newsletters are having such a great moment in the sun right now. Because they’re designed for that one-sided, parasocial type of community. <Mmhmm.> 

      Where if, you know, we’re recording a podcast right now, it’s just a conversation between me and you. And, yes, hopefully, thousands of people will be listening to it later. But that we’re, sort of, like, doing this in public — meaning that we’re publishing our conversation — but we’re not actively interacting with the audience that might be listening to us. And similarly, with a newsletter, I can write this long-form post and share my thoughts in a higher-context situation. I assume or hope that most of the people subscribing have some context for who I am. And then I can, kind of, send it out, and people can read it on their own time. It’s not the same thing as when I tweet something out, and then literally anybody with an internet connection… <Right.> I made a public tweet — can see it and respond to it and pass it around and do whatever they want with it.

      Crisis of the commons

      Sonal: And so, to summarize, the clubs are the projects — the spaces with high contributor growth and low user growth, like these private messaging groups. The stadiums are, like, the projects with a low contributor growth and high user growth, like these newsletters and podcasts. I really think, Nadia, one of the best things about your book is this framework of the federations, the club, the stadiums, the toys — because you dehomogenize this phrase “open source and community.” And then it, correspondingly, gives people frameworks for what that means for how you build, support, nurture that. 

      So, I’m now gonna switch to asking you some practical questions about that. Platforms are having a moment right now, for better or worse. It’s one of the reasons that we also are very excited about crypto and talking about communities. And I want to talk about the tragedy of commons and the work of Elinor Ostrom, who is definitely having a moment. Right now you and a lot of other people I know have been citing her work. One of our former partners, Jesse Walden, wrote a post about cooperatives as an analogy for crypto networks. And he cited some of the conditions that she cites in governing the commons — and then you yourself summarize the conditions. Can you, A, tell me what those are — B, tell me why you think this is important, and then help me connect the dots for how that matters practically?

      Nadia: Sure. So, Elinor Ostrom was a researcher who became well known for her work around trying to understand why the tragedy of the commons occurs, and how we might avoid it or move around it. Tragedy of the commons just sort of being that — if everyone has access to a shared resource. You can imagine a fishery or a forest — anyone can cut down wood in the forest. But if everyone does that, and just, kind of, does what they want for themselves, then eventually that forest is gonna be depleted unless it is managed in some shape or form. 

      And so, tragedy of the commons is this concept from ages ago that is, maybe, one possible outcome of the commons. But it’s almost like when people talk about the commons, they always talk about tragedy of the commons — as though you can’t have, like, a non-tragedy of the commons. And so, Elinor Ostrom is basically looking at — what are situations where commons are being sustainably self-managed. And she did decades of research looking at these, like, fisheries and forests, and just, like, different examples of commons, and then documenting what she found and summarizing them into principles for — if you are in the situation where you have this shared resource, how can you manage it without everyone just sort of taking for themselves.

      And so, I talked about her conditions in the book a bit, and the ones that I’ll point to that are most relevant for this conversation are — this idea that in order to have a well-managed commons, you do need to draw boundaries around membership. It needs to be clear who is allowed to appropriate from the commons and who isn’t. And then with that, there are all these implications of, well, what does it mean to be a member of the commons? A couple of things that I’ll highlight are, one, this idea that you have high context for your interactions with other people that are also members.

      Sonal: Yep, that creates trust. That’s what creates trust. It’s just like in a company. They say the best advice you can give to any team or any fast-growing group is the more shared context, the more trust you have — because you can do more shortcuts together in your work.

      Nadia: That’s right. High context, high trust is a really important implication of having these clear membership boundaries. And then the other thing I’ll point to is the idea of having a low discount rate — which is just saying that if you’re a member of this community, you expect to be around for a while.

      Sonal: Sorry, what do you mean by low discount rate?

      Nadia: Low discount rate is just this idea that if you’re invested in the community for a long period of time, you’re not planning on hopping in, saying something rude — if this applies to online communities — and then just, like, hopping out and disappearing. You’re like, “I’m stuck here. I need to, like, actually learn how to work with everybody else.”

      Sonal: Right. It’s actually, kind of, like, skin in the game.

      Nadia: Yes. In order for a commons to function in this healthy way, you need to have these underlying conditions of high context, of having skin in the game, of having clearly defined membership, among a bunch of other things that I won’t get into here. Her work is finding, I think, renewed appeal right now — especially because people are trying to answer these questions in open source and in online communities elsewhere, of just, like — how can communities self-manage and not implode over time? 

      There’s so much that is relevant about her work to today. I think it mostly applies, though, to the concept of clubs — clubs basically being this commons, where everyone has a stake in what they’re creating. If we think about a stadium — a creators’ community that is on a very public social platform — the whole concept of the commons kind of breaks down, right? Like, I mean, if I’m tweeting in public, anybody can read what I’m saying. And until recently, as Twitter’s now making it possible for people to limit comments on their tweets, and things like that — but for the most part, like, anybody can just, like, comment on my tweet and jump in.

      And so, understanding, I think, both what her theory of the commons was — and why it doesn’t really apply to today — can help answer some of these questions about — is it okay to have common threads that are entirely open to everyone? What are the problems that might arise from that? And then what can we do to actually limit interactions from outsiders, so that the people that are most involved or have most skin in the game can actually get stuff done? 

      And it’s a difficult thing to talk about, because it can be taken as gatekeeping or trying to keep other people from participating. It’s just, like, a touchy subject. I can’t say that we should just close off the boundaries entirely. And I think this gets to, again, the idea of — you can have things that are public but not participatory. It’s okay to make software that anyone can use. That doesn’t mean that everybody who uses your software can also participate in its production. So, it’s really just about finding kind of, like, a middle ground there.

      Sonal: You have so many great analogies in your book — that sometimes this is more like directing air traffic, given the flood of abundance we have on our internet today. So, on that front, I’m gonna ask you just a couple of quick questions on the practical front. Let me do this lightning round style. What is one key piece of advice you might have for community managers?

      Advice for platform management

      Nadia: For community managers — first thought is, just know what kind of community you’re in charge of. Which is where I think it’s helpful to have a set of different models in your mind of — are you actively trying to bring in lots more contributors? It’s okay if you’re not. Some communities do better on contributor retention, and less so on contributor growth. And that’s totally fine. Or is it the kind of thing where there is a lot of work that needs to be done? And do you think you stand a chance of recruiting more people? Then go recruit more people. 

      It’s fine to have a community that isn’t super high growth but is stable. It’s fine to have a community that is extremely high growth, where you’re trying to bring in lots of different kinds of members and make it this really bustling, kind of, federation-style community. It’s fine to have just one person that is, sort of, standing up in front of a crowd. That is a community in its own form, but it just requires different sorts of strategies to figure out how to manage it.

      Sonal: You’re basically saying — know the difference of whether you have a club or a stadium. And, by the way, you quoted that person talking about the Newtonian phase. Who knows? There might be times when you can have both in one place, so that can change. And then how about advice for platforms?

      Nadia: For platforms, I would say — take your creators seriously and the responsibility and the relationship that you have to them. And what I mean by that is that platforms are really the only place to create these closed status economies that enable creators to continue doing their work for however long they want, and to open up all these amazing opportunities for creators. And sometimes that doesn’t directly happen on your platform — as in, like, maybe it’s not that they can raise money directly on your platform, but it is important to make their status legible to others so that they can take that clout and that reputation and actually, like, shop it around to turn it into other opportunities.

      Sonal: I might even add that crypto is great for that, because that’s where you can actually port some of your currency — your reputation currency — and prominence in a way that — like, in blockchains. And then, finally, advice for leadership and/or communication tools. Because we’ve talked a lot about — I think a lot of times people make the mistake of talking a lot about collaboration and coordination. But they don’t often talk about the communication part of things. And this is particularly heightened in our remote world. So, any advice you have on the communication tools side and then anything for the leadership side?

      Nadia: More so the leadership side. On the leadership side, I think it’s, again, about knowing your community and not being afraid to be decisive. A lot of communities that I’ve looked at have suffered in one direction or the other of — either being overly deferential to their community and trying to treat it like this pure democracy, when really the community’s size or shape is just so unwieldy that that’s not really possible. And so it is okay to say, “These are the decisions that are being made. And we don’t have to make this — bring this to a vote every time we want to decide what we want to do.”

      And maybe also on the flip side, that depending what type of community you are overseeing, there are ways to bring those active voices and contributors into leadership and encourage more people to participate. But, again, it depends on whether you’re on this, like, high growth side or low growth side. On the communication tools side, I think this idea that separating the ideas of public and participatory is just gonna lead to a lot of really interesting things happening in the near future. Just getting playful with the idea that a community does not mean that everyone is participating at equal volume and, you know, shouting at each other — because we’ve seen with, like, every social platform, that gets pretty hard at scale.

      And so, like, as we’re creating new things today, it’s fun to think about the opportunity we have in front of us to actually design from scratch about — in thinking, like, how would we have these sort of scaled social interactions? And so disambiguating the idea of public and participatory can just lead to really fruitful new ways of communicating.

      Sonal: This reminds me — one of my favorite quotes from Questlove. This is from his book, “Creative Quest.” He basically writes that when you make work, you are the creator, but also the eventual audience — which I think is such a powerful idea. There’s, like, so many different ways to interpret that. What I love about your book is that it’s not a grand theory of everything. It ties together lots of different themes together in a really meaningful way. But you can also tease them apart regardless of your vantage point, whether you’re a creator, open source, business — however, you might define a company or any form of coordination and collaboration.

      I also appreciate, given that you were kind of newer to the community — compared to, sort of, the first, early generations — that you don’t bring this sort of chip of nostalgia, and, sort of, come at it from a very first principles approach, and just sort of really bring all your insights together. So I just want to thank you for “Working in Public: The Making and Maintenance of Open Source Software.” I’m gonna add my own personal subhead, which is — and many, many other kinds of orgs. So don’t just not read it if you don’t think it’s about you, because open source is everyone. Thank you for joining the “a16z Podcast.”

      Nadia: Thank you. It was a pleasure to be here.

      • Nadia Eghbal is an independent researcher exploring how the internet enables creators. Previously she focused on the production of open source software and worked at Substack. She wrote the book Working in Public.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      GPT-3, Beyond the Hype

      Frank Chen and Sonal Chokshi

      In this special “2x” explainer episode of 16 Minutes — where we talk about what’s in the news, and where we are on the long arc of various tech trends — we cover all the buzz around GPT-3, the pre-trained machine learning model that’s optimized to do a variety of natural-language processing tasks. The paper about GPT-3 was released in late May, but OpenAI (the AI “research and deployment” company behind it) only recently released private access to its API or application programming interface, which includes some of the technical achievements behind GPT-3 as well as other models.

      It’s a commercial product, built on research; so what does this mean for both startups AND incumbents… and the future of “AI as a service”? And given that we’re seeing all kinds of (cherrypicked!) examples of output from OpenAI’s beta API being shared — from articles and press releases and screenplays and Shakespearean poetry to business advice to “ask me anything” search and even designing webpages and plug-ins that turn words into code and even does some arithmetic too — how do we know how good it really is or isn’t? And when we things like founding principles for a new religion or other experiments that are being shared virally (like “TikTok videos for nerds“), how do we know the difference between “looks like” a toy and “is” a toy (especially given that many innovations may start out so)?

      And finally, where are we, really, in terms of natural language processing and progress towards artificial general intelligence? Is it intelligent, does that matter, and how do we know (if not with a Turing Test)? Finally, what are the broader questions, considerations, and implications for jobs and more? Frank Chen (who’s shared a primer on AI/machine learning/deep learning as well as resources for getting started in building products with AI inside and more) explains what “it” actually is and isn’t; where it fits in the taxonomy of neural networks, deep learning approaches, and more in conversation with host Sonal Chokshi. And the two help tease apart what’s hype/ what’s real here… as is the theme of this show.

       

      image source: Gwern.net 

      Show Notes

      • What is GPT-3, how do developers access it [3:56], and how is it different from other machine learning tools? [6:52]
      • Discussion of how to categorize GPT-3, how it learns [9:57], and where it fits into the AI big picture [13:43]
      • Real-world applications and scalability [16:20]
      • How existing technology companies may respond to tools like GPT-3 and further iterations [19:16]
      • Discussion of how people may work with tools like GPT-3 in the future, and how it could transform the workplace [20:54]
      • Ethical concerns around stereotyping and racism in AI [24:34]
      • The need for a new Turing test for AI [26:47] and predictions for the future [30:18]

      Transcript

      Sonal: Hi, everyone. Welcome to this week’s episode of “16 Minutes.” I’m Sonal, your host, and this is our show where we talk about the headlines, what’s in the news, and where we are on the long arc of tech trends. We’re back from our holiday break, and so this week we’re covering all the recent and ongoing buzz around the topic of GPT-3, the natural language processing-based text predictor from the San Francisco research and development company, OpenAI.

      They actually released their paper on GPT-3 in late May, but only released their broader commercial API a couple of weeks ago. So, we’re seeing a lot of excitement and activity around that, in particular, although it’s all being called GPT-3. So, we’re going to do one of our explainer episodes. It’s a 2x explainer episode going into what it really is, how it works, why it matters, and broader implications and questions while teasing apart what’s hype, what’s real, as is the premise of the show. But before I introduce our expert, let me just quickly summarize some of the highlights.

      So, while GPT-3 is technically a text predictor, that actually reduces what’s possible because, of course, words and software are simply the encoding of human thought — to borrow a phrase from Chris Dixon — which means a lot more things are possible. So we’re seeing, and note these are all cherry-picked examples — believable forum posts, comments, press releases, poetry, screenplays, articles, someone even wrote an entire article headlined “OpenAI’s GPT-3 may be the biggest thing since Bitcoin,” and then revealed midway that he didn’t actually write the article, but that GPT-3 did.

      We’re also seeing strategy documents, like for business CEOs and advice written entirely in GPT-3. And not just words, but we’re seeing people design, using words, to write code for designing websites and other designs. Someone even built a Figma plugin — again, all of it showing the transmutability of thoughts to words, to code, to design, and so on. And then someone made a search engine that can return answers and URLs in response to “ask me anything,” which as anyone who’s been in the NLP space knows. I was at PARC when we spun off Powerset, back in the day, and that’s always been sort of a holy grail of question-answering, which you know all about too having worked in this world, Frank.

      And now, let me introduce you — our expert in this episode. Frank Chen has written a lot about AI, including a primer on AI deep learning, and machine learning, a pulse check on AI, what’s working, what’s not, a microsite with resources for how to get started practically and do something with your own product and your own company, and then reflecting on jobs and humanity and AI working together. You can find all of that on our website.

      Frank, to start things off, what’s your favorite example of GPT-3 so far? Mine is founding principles for a religion written in GPT-3. I’d love to hear your favorite and also your quick take on why the excitement — to start us off before we dig in a bit deeper.

      Frank: My favorite out of the whole thing is it’s doing arithmetic. So, if you ask it what’s 23 plus 67, like just arbitrary two-digit arithmetic, it’s doing it. This is a natural language processing model. And so, basically, it got trained by feeding it lots and lots of text. And out of that, it’s figuring out — we think — how to do arithmetic, which is very, very surprising, because you don’t think that, like, exists in text. The excitement potentially is promising signs of, you know, progress towards general artificial intelligence.

      So, today, if you want to do very highly accurate natural language processing, you build a bespoke model. You have your own custom architecture, you feed it a ton of data. What GPT-3 shows is that they train this model once and then they throw it a whole bunch of natural language processing tasks — like fill in the blank, or inference, or translation. And without retraining it at all, they’re getting really good results compared to finely-tuned models.

      What actually is GPT-3?

      Sonal: Before we even go into teasing apart what’s hype, what’s real, let’s first talk about the “it.” What is GPT-3?

      Frank: So, we have two things. One, we have a machine learning model. GPT is actually an acronym — it stands for Generative Pre-Trained transformer. We’ll go through all those in a sec. But thing one is, we have a pre-trained machine learning model that’s optimized to do a wide variety of natural language processing tasks, like reading a Wikipedia article and answering questions from it; or guessing what the ending of a story should be; or so on and so on. So we have a machine learning model. The thing that people are playing with is an API that allows developers to essentially ask questions of that model. So, instead of giving you the model and you program it to do what you want, they’re giving you selective access via the API.

      One of the reasons they’re doing this is that most people don’t have the compute infrastructure to even train the model. There’s been estimates that if you wanted to train the model from scratch, it would cost something like $5 to $10 million of cloud compute time. That’s a big, big model, and so, like, they don’t give out the model. And then two, the controversy around this thing when they released the first version was they were worried that if they just gave the raw model out, people would do nefarious things with it — like generate fake news articles that you would just, like, saturate, bomb the web — and so they were like, look, we want to be responsible with this thing, and so, we’ll gate access via API so then we know exactly who’s using it. And then the API can be a bit of a throttle on what it can and can’t do as well.

      Sonal: Right. Well, while helping them learn. And just as a reminder, APIs are application programming interfaces. We’ve talked a lot about them on the podcast, and people who want to learn more can go to a16z.com/api to read all our resources, explainers. There’s so much we have on this whole topic. But the key underlying idea — and this goes to your point about the cost of what it would take if you were trying to build this from scratch — is APIs give developers and other businesses superpowers because they lower the barrier to entry — in this case, for anyone being able to use AI who doesn’t necessarily have a whole in-house research team, etc. And so, that’s one of the really neat things about the API.

      But I do want to correct one misconception the folks out there aren’t aware of when it comes to GPT-3. What they’re describing as GPT-3, they’re actually playing with OpenAI’s API, which is not just GPT-3. Obviously, some of the technical achievements of GPT-3 are in the API, of course, but it’s a combination of other things. It’s like a set of technologies that they’ve released and it’s their first commercial product, in fact. So, that’s just to give people a little context on what the “it” is and isn’t there. Let’s go ahead and go a level deeper into explaining what it is. In their paper, they describe it simply as an autoregressive language model. Can you share what it is and kind of the category this fits in?

      What categories does GPT-3 fit into?

      Frank: Yeah. So, the broad category of things it fits into — it is a neural network, or a deep neural network. And architectures basically talk about the shape of those networks. At the highest level, visualize it as something comes in on the left, and then I want something to shoot out on the right side — and in between is a bunch of nodes that are connected to each other. And the way in which those nodes are connected to each other and then the connection weights, that’s essentially the neural network. GPT-3 is one of those things. Technically, it’s called a transformer architecture. This is an architecture for neural networks that Google introduced a few years ago. And it’s different than a convolutional neural network, which is great for images. It’s different than a recurrent neural network, which is good for simple language processing. The way the nodes are connected to each other results in it being able to do, essentially, computations on large sentences <Yes.> filled with different words and doing it concurrently instead of sequentially. So, RNNs, which were the former state-of-the-art on natural language processing, they’re very sequential. So, they’ll kind of go through a sentence a word at a time…

      Sonal: Recurrent, right?

      Frank: Exactly. These transformer networks can basically, sort of, consider the entire sentence in context while it’s doing its computations. One of the things that you classically have to do with natural language processing is you have to disambiguate words. “I went to the bank” — that could mean I want to go withdraw some money, or it can mean I went right up to the edge of the river — because we have ambiguity in these words. The natural language processing system needs to figure out, well, which sense of bank did you mean? And you need to know all the other words around that sentence in order to disambiguate it.

      And so, these transformers consider large chunks of text in trying to make that decision all at once instead of sequentially. So, that’s what the transformer architecture does. And then what OpenAI has been doing is basically transforming this type of neural network, with the transformer architecture, on larger and larger datasets. Conceptually, think of it as you’ll have it read Wikipedia, and think of that as generation one. Generation two is, I’m going to have it read Wikipedia and all of the open-source textbooks that I can find. This generation, they trained it on what’s called common crawl. It’s kind of the same thing that Google uses to search and index the internet. There’s an open-source version of that. Think of it as — robots go onto every webpage, they gather the text, and now we’re using that as the training set for GPT-3.

      Sonal: Yeah. Something like half a trillion words, I believe.

      Frank: Yeah. It’s a crazy number of words. And then this thing has two orders of magnitude more than the previous attempts, that’s something like 175 billion parameters. For the purposes of this conversation, a way of measuring the complexity of a neural network.

      Sonal: Right. GPT-2 had 1.5 billion.

      Frank: And in between GPT-2 and 3, Microsoft did one that was 17 billion, right? So, like, there is a bit of an arms race here going on, which is, like, how big are your neural networks?

      How GPT-3 learns

      Sonal: What does it mean, because the paper’s called “Language Models are Few-Shot Learners.” And I remember this movement in one-shot learning where you can learn on very few examples, but honestly, what you just described to me sounded like almost a trillion examples, when you think about what it’s ingesting as an input. So, can you actually explain what few-shot even means in this context?

      Frank: Yeah. So, first, they trained this model on the internet. Basically, what came in as input on the left side was reams and reams and reams of text — all the text they could get their hands on, and they cleaned it a little. And so, this is very traditional deep learning. It is not itself a zero-shot or a few-shot approach. It’s deep learned, which means I have incredible amounts of input text. What they mean in the context of this paper around no-shot and few-shot is, the model can perform a variety of natural language processing tasks. So, a good example of it is analogies — king is to queen, as water is to what, right?

      In the context of the system, what you can do is you could give it an example of that, and they call that one-shot — which is, I’m going to give you an example of an analogy that’s completely filled out, and then I want you to fill out more analogies. Another task would be — pick the right ending of a story, and I will give you one example with the correct answer. So, I’m just going to give it to you once. Now, typically what happens when you do traditional neural network learning — you take an example, you give it to the system, and you tell the system the right answer. The system uses that right answer to basically readjust the neural net. It’s called backpropagation. And the theory is that, as it adjusts the weights inside the neural network, it will get that answer more correct the next time it sees it.

      And so, everything up until this point has basically been — if I give you enough examples, I’m going to be able to tell whether that picture has a hot dog in it or not. I will be able to generalize the features of a hot dog, and I will basically deduce hot-dogness if you just give me enough pictures and you tell me, hot dog or not. What’s going on here is they train this model once, and then they give it one example — that example doesn’t adjust the weights of the model. It really just primes the system to basically prepare it to answer this type of question. So, you basically tell it, look — I want you to work on, fill in the blank, and I’m gonna give you one or a few examples (few-shot) of this, and then we’ll go from there. But those examples that you give it don’t adjust the weights of the model. It’s one model to rule them all. And this is kind of how humans learn. They don’t need to see 1,000, 10,000, 100,000 examples of hot dogs before they can start reliably telling whether it is a hot dog or not.

      Sonal: It’s like how children learn language.

      Frank: Yeah, exactly. Babies, before they can say cat and dog, can recognize the difference between cats and dogs — they didn’t see a million of them, right? In fact, they can’t say the words dog and cat yet. And so, maybe something like this is going on in the brain, which is you have this sort of general processor, and then it instantly knows how to adapt itself to solve a lot of different problems, including problems it had never seen before. And so, I’m going to go back to my favorite example of, like, what GPT-3 was used for. Like, how in the world did it deduce the rules for two-digit arithmetic by reading a lot of stuff? And so, maybe this is the beginnings of a general intelligence that can rapidly adapt itself. Now, look, I don’t want to get ahead of myself. It falls apart on four-digit arithmetic. And so, it’s not generally smart yet, but the fact that it got all of the two-digit addition and subtraction problems right by reading text, like, that’s crazy to me.

      Fitting GPT-3 into the AI big picture

      Sonal: The general takeaway is that it does some complicated things really well, and some really easy things really badly, and this is actually true of most AI. The researchers have a huge section on limitations where, “GPT-3 samples can lose coherence over sufficiently long passages, contradict themselves, and occasionally contain non sequitur sentences or paragraphs.” Now, of course, as an editor, that made me laugh because that’s also true of human writing. <laughter> So, I was like — okay, this is also true about the writing I’ve seen and edited, so I don’t know who’s talking here. Help me tease apart where we really are in this long arc. I’m having a hard time knowing what’s real, what’s not. Like, help me kind of understand what is this thing, really, at this moment in time.

      Frank: So, we have the most sophisticated natural language processing pre-trained model of its kind. The natural language processing community has basically divided the problem of understanding language into dozens and dozens of sub-tasks. And task after task after task, GPT-3 goes up against the state-of-the-art, the best performing system. And basically what the paper does is lay out, okay, here’s where GPT-3 is approaching state-of-the-art, here’s where it’s far away from state-of-the-art. And that’s basically all we know, is — compared to state-of-the-art techniques for solving that particular natural language processing task, how does it perform? We’re really in the research domain. <Right.> So, if you were to ask me, can I build a startup on it? Can I build the world’s best chatbot on it? Can I build the world’s best customer support agent on it?

      Sonal: I was going to ask you that.

      Frank: Yeah, I think it’s really too early to tell whether you can build any of those things. The hope is that you could, and long-term, really, the hope is, having built a model like this and exposed an API, you could take any Silicon Valley startup that wants to solve a text problem — chatbots, or pre-sale support, or post-sales customer support, or building a mental health app that talks to you. All of those things will get dramatically cheaper and faster and easier to build on top of this infrastructure.

      If this works, you have this generally smart system that’s already been trained, then you show it a couple examples of problems that you want to solve, and then it will just solve them with very high accuracy. All you have to do, as a startup or a programmer, is to say, “Hey, look, I’m going to give you a couple of examples of the type of problem that I want solved.” And then that priming is going to be enough for the system to get very accurate results. And, in fact, sometimes better results than if you had built the model and fed it the data sets yourself. So, that’s the hope, but we just don’t know yet.

      Use and scalability

      Sonal: That’s a really good reminder because they themselves are like, this is early days, it’s research, there’s a lot of work to be done — but it’s also really exciting, as you’re saying, because this is one of the most advanced natural language models we’ve seen. So, the question I have then, on the startup and building side — what would it take to — what are the kinds of considerations to make it more practical and scalable? I mean, for one thing, the size — you described how the transformer has this ability to sort of comprehend so much at once without doing it in kind of this RNN model, but the trade-off of that is that it’s so slow, or be able to fit on a GPU. So, I’d love to have a quick take from you on, what are the things that need to happen to make something like this more usable, etc.

      Frank: I think what’s going to need to happen is that the OpenAI product team is going to have conversations with dozens and dozens of startups that are using their technology. And then they successfully refine the API and improve the performance, and set up the security rules and all of that, so that it becomes something as easy to use as say, Stripe or Twilio. Stripe or Twilio are very straightforward — send a text message or process this payment. This is a lot more amorphous, which is, “Hey, I can do SAT analogies. How’s that relevant for my startup?” Well, there’s a bit of a gap there, right? You have a startup that’s like, “Hey, I need my documents summarized,” or, “I need you to go through all of the complaints we’ve ever gotten and give me product insight for product managers.” And so, there’s basically a divide between there that needs to be closed over time.

      Sonal: Right. So, what does this mean with the data world? Because one really interesting [thing] to me is, on one hand, APIs give you superpowers — kind of democratizing things. On the other hand, it kind of makes things a bit of a race to the bottom then, because then you have to differentiate — kind of private, proprietary, these other elements. So, do you have thoughts on what that means?

      Frank: Yeah. I mean the hope for something like a GPT-3 is that it’s going to dramatically reduce the data gathering, cleansing, cleaning process — and, frankly, building the data model as well, your machine learning model. So, let me try to put it in economic terms. Let’s say we put $10 million into a Series A company, and then $5 million of it goes to getting data and cleaning it and hiring your machine learning people, and then renting a bunch of GPUs in Amazon or Google or Microsoft, wherever you do your compute. The hope is that if you could stand on the shoulders of something like GPT-3 — and it’ll be a future version of it — you would reduce those costs from $5 million to $100,000.

      You’re basically making API calls and the way you program “this thing” is you just show it a bunch of examples that are relevant to the problem that you’re trying to solve. So, you show it texts where you had a suicide risk and you don’t need to show it a bunch because it’s pre-trained — and you show it a new text that it hasn’t seen before and you ask it, “What is the risk of suicide in this text exchange?” The hope is that we can dramatically reduce the costs of gathering that data and building the machine learning models. But it’s really too early to tell whether that’s going to be practical or not.

      Sonal: So we know what it means for startups, but how do the incumbents respond in that kind of a world? But it seems almost inevitable that the big players — there might be an AWS potentially, right, that could, you know, make this a given in their services — like this kind of bigger question around this business model of AI as a service.

      Frank: Yeah. So, the first thing I’ll say is this is OpenAI’s first commercial product, which is interesting, right? Recall that OpenAI started as a research institution, so we’ll sort of see what the pricing is. If this works, the scenario that I described earlier, which is — dramatically reduce the time it takes to build a machine learning inside product — then all of the public cloud providers and other startups will offer competing products because they don’t want to let OpenAI just take all of the, sort of, text understanding ability of the internet, right?

      Google Cloud and Microsoft and Amazon and Baidu and Tencent, like they’re all gonna say, “Hey, look, I can do that too — build your application on me.” Now, I will say that because of the large costs of training the model — so I’d mentioned estimates ranging from $5 million to $10 million to train this thing once — and obviously, they didn’t train it once to get to where they were, they trained it multiple times as they did the research process. And so, this is not going to be for the faint of heart. It’s going to come on the back of a lot of money with very skilled scientists using enormous infrastructure. But to the extent that this product works, then you’re going to have very healthy competition among all of the incumbents. You might even have new players who’ll figure out a different angle on it.

      Working with machine learning

      Sonal: You know, it’s really fascinating watching the people who have access. And basically, the recurring theme is that it’s not like plug and play, it’s obviously not built and ready for that yet. The prompt and the sampling hyper-parameters matter a lot. Priming is an art, not a science. So, I’m curious for where you think the knowledge value is going to go in the future. What are the sort of — the data scientists of the future going to look like for people who have to work with something like this? Now, granted the models are going to evolve, the API will evolve, the product will evolve — but what are the skills that people need to have in order to really do well in this world coming ahead?

      Frank: It’s really too early to tell, but it is a fundamentally different art of programming, right? So, if you think of programming to date, it’s basically — I learn Python, and I learn to be efficient with memory, and I learn to write clever algorithms that can sort things fast. That’s well-understood art, thousands of classes, millions of people know how to do that. If this approach works, basically, there is this massive pre-trained natural language model, and the programming technique is basically I show you a couple of examples of the tasks that I want you to perform — it’ll be about what examples do I show you, and in what form? And do I show you the outliers, or do I show you some normal ones, right? And so, if this approach works, it’ll all be about — how do you prime the model to get the best accuracy for the real-world problems you actually want your product to solve? Programming becomes — what examples do I show you, as opposed to how do I allocate memory and write efficient search algorithms? It’s a very different thing.

      Sonal: Vitalik Buterin, the inventor of Ethereum, described this when he was observing some of this buzz around GPT-3 that, “I can easily see many jobs in the next 10 to 20 years changing their workflow to ‘human describes; AI builds; human debugs.’” There’s a lot of speculation about how this might affect jobs. It can displace customer support, sales support, data scientists, legal assistants, and other jobs like that are at risk. Do you have thoughts on the labor and jobs side of this — like just sort of the broader questions and concerns here?

      Frank: The way I think about this generally — and informed a lot by Erik Brynjolfsson and other people — so if you think about a job as a set of tasks, some tasks will get automated, and then some tasks will be stubbornly hard to automate, and then there’ll be new tasks. And so, think of jobs as sort of an ever-changing bundle of tasks, some of which are performed by humans today, some of which will get automated, and then there are new tasks. And so what Vitalik describes — if this AI stuff works, being able to prime the AI system with the right examples, and then being able to debug it at the end — those are two new tasks. No human on the planet gets paid to do that outside of AI researchers today. But that could be mainstream knowledge work in 10 years, which is — you pick good examples, and then you debug it at the end. So, you have these brand new tasks that are generating economic value and people get paid for them, that didn’t exist before.

      Sonal: I find it very fascinating what you said, by the way, because what it also means to me is it becomes more inclusive for more people to enter the worlds that might have been previously closed off to a certain class of type of programmers, or people who have certain technical skills, because — let’s say you’re very good at describing things, and it’s more of an art than a science, and you’re very good at sort of fiddling with and hacking at things, you might be better off than someone who went through like years and years of elite Ph.D education at tuning something than someone else.

      Frank: I think the machine learning algorithms will invite more people who would otherwise be discouraged into pursuing careers, in careers they wouldn’t have naturally risen to the top of. So I think you’re right.

      Ethical concerns and safeguards

      Sonal: What do you make of the concern — there was concern that GPT-3, these answers that it gave, that it predicted, were rife with racism or stereotypes. What do you make of the data issues around that?

      Frank: Okay. We’re going to feed it every piece of text on the internet and then we’re going to ask it to make generalizations. What could possibly go wrong? A lot could possibly go wrong. If you look at the heart of this system, it’s basically, I’m trying to guess the next word. And the way I make my guess is, I go look at all the documents that have been written ever and I ask, what words are most likely to have occurred in those documents, right? You’re going to end up with culturally offensive stereotypes. And so, we need to figure out — how do we put the safety rails? How do we erect the APIs? I’m glad the OpenAI researchers and the community around them are being very careful about this because we obviously have to. How do we basically teach it the social norms we want it to emit, as opposed to the ones that it found by reading text?

      Sonal: Another whole philosophical sidebar, but really important is, if you think about the internet as the sum total of human knowledge, then other things that reflect many of the realities in the world, which are atrocious and awful in many cases. The flip side of it is, it’s a lot harder to change the real world and people and behavior and society and systems, but probably a hell of a lot easier to change a technical system and be able to do certain things. So, to me, what’s implicit in what you said is that there’s actually a solution — I don’t mean to be solutionistic, but that’s within the technology that you don’t necessarily get from IRL, in real life.

      Frank: Yeah, that’s exactly right. And if it were in algorithm land, so to speak, where we are, right, GPT-3 and its descendants — let’s say GPT-17 gave you a text document, right? It wrote a text document for you. You could take that document and put it through whatever filter that you wanted, right, to filter out sexism or racism, and that layer could be inspectable and tuneable to everybody. You didn’t know how GPT-17 came up with its recommendations, but you have this safety net at the end, which is you can filter out things that you don’t want. So, you have the second step that you can actually put into your system. You don’t have to depend just on the first thing, you can catch that at a subsequent stage.

      Updating the Turing test

      Sonal: Right. And you can have sort of a system of checks and balances. So a broad meta question — one of my favorite posts was from Kevin Lacker, and he basically gave GPT-3 a Turing test, and he tested it on these questions of common sense, obscure trivia, logic. And one of the things he observed is that, “We need to ask it questions that no normal human would ever talk about.” And so, he said, if you’re ever a judge in a Turing test, make sure you ask some nonsense questions and see if the interviewee responds the way a human would. Because the system doesn’t know how to say I don’t know, and this goes at this question of what does a Turing test tell us? And there’s been a lot of work, as you know, over the years about the modernization of the Turing test — like in 2016, Gary Marcus, our friend, Gary Marcus, Francesca Rossi, and Manuela Veloso published an article “Beyond the Turing Test” in “AI Magazine.”

      Barbara Gross of Harvard wrote a piece called “What Question Would Turing Pose Today?” in “AI Magazine” in 2012. And she basically starts by saying that in 1950, when Turing proposed to replace the question “can machines think?” with the question “are there computers which would do well in the imitation game?” — at the time, computer science wasn’t a field of study. You know, Claude Shannon’s theory of information was just getting started. Psychology was just only starting to go beyond behavior. And so, what would Turing ask today? He’d probably propose a very different test. And so, the question I really wanted to ask you is, how do we know if the thing is measuring what it’s supposed to measure, or answering what it’s supposed to answer, or that it’s getting smarter, I guess?

      Frank: This is more a philosophical question than an engineering question. So, why don’t I say what we know, and then I’ll widely speculate on the other stuff?

      Sonal: That’s great. That’s life and science, so go for it.

      Frank: Exactly. So, basically, if you read the paper, you’ll see that it compares GPT-3’s performance against various other state-of-the-art techniques on a wide variety of natural language processing tasks. So, for instance, if you’re asking it to translate from English to French, there’s this thing called the BLEU score. The higher the BLEU score, the better your translation. And so, every test has its measure. And so, what we do know is we can compare GPT-3 performance versus other algorithms, other systems. What we don’t know is, how much does it really understand? So, what do we really take away from the fact that it aced two-digit arithmetic? Like, what does that mean? What does it understand of the world? Does it get math? Let’s say you had a system that was 100% accurate on every two-digit arithmetic problem that you ever gave it. It’s perfect at math, but it doesn’t get it. Like, it doesn’t know that these numbers represent things in the real world, but what does that mean to claim that it doesn’t get it? That’s a philosophical question.

      Sonal: Right. It’s philosophical because the question then becomes — does it even matter if it comes to applying things practically? Because I think about this from the world of education, you know, there’s a big focus on metacognition and the awareness of knowing what you know and don’t know. But at a certain point, if the kid is doing well on the test and the test is applicable to the world, and they can basically survive and do well, does it even matter if they really understood what arithmetic really means, as long as they can solve the problem when you go to the store, that I give you a dollar, I get 5 cents change back? You know what I mean?

      Frank: That’s exactly right. And if you generalize that out to other tasks that humans solve in the real world, imagine you just got good at 100 and then 1,000 and then 10,000 of these tasks that you have never seen before. Let’s say descendants of GPT-3 got that good at a wide variety of language tasks — what does it mean to insist, but it doesn’t get the world, it doesn’t get language, right? <laughter>

      Predictions about future implications

      Sonal: Yeah. That’s fantastic. I’d love to get sort of your perspective on how we think about this broader arc of innovation that’s playing out here. Daniel Gross called GPT-3 screenshots the TikTok videos of nerds. And there’s something to that — it’s kind of created this inherent virality. So, I’m curious for your take on that. On the one hand, some of the most important technologies start out looking like a toy. Chris Dixon paraphrased a really important idea from Clayton Christensen about how disruptive innovation happens. But a lot of the people who are researchers really emphasize — this is not a toy, this is a big deal.

      Frank: There are a lot of TikTok-ish-like videos that are coming out of the whole playground, which is basically a place where you can try out the model. And on the one hand, people are saying it’s a toy because they’re in the sandbox and they’re basically having fun feeding it prompts. Some of those examples are actually really good, and some of those are, like, comically bad, right? So, it feels toy-like. The tantalizing prospect for this thing is that we have the beginnings of an approach to general intelligence that we haven’t seen us make this much progress on before, which is — today if you wanted to build a specific system for a specific natural language processing task, you could do that. Custom architecture, lots of training data, and lots of hand-tuning and lots of, like, Ph.D time.

      The tantalizing thing about GPT-3 is, it didn’t have an end-use case in mind that it was going to be optimal for, but it turns out to be really good at a lot of them, which kind of is how people are. You’re not tuned to, like, learn polka or double-entry bookkeeping, or learn how to audio-edit a podcast — like, you didn’t come out of the womb with that, but your brain is this general-purpose computer that can figure out how to get very, very good at that with enough practice and enough intentionality.

      Sonal: Well, it’s really great that you use the word tantalizing because if you remember the Greek myth root behind it, Tantalus was destined to constantly get this like tempting fruit dangling above him as punishment. And it was so close yet so out of reach at the same time. So, bottom line it for me, Frank.

      Frank: It’s tantalizing, right? Now, look, there’s a limit to how big these models can get and how effective the APIs will be once we sort of, you know, unleash them to regular programmers. But it is surprising that it is so good across a broad range of tasks, including ones that the original designers didn’t contemplate. So, maybe this is the path to artificial general intelligence? Now, look, it’s way too early to tell. So, I’m not saying that it is, I’m just saying it’s very robust across a lot of very different tasks, and that’s surprising, and kind of exciting.

      Sonal: Thank you so much for joining this episode of “16 Minutes,” Frank.

      Frank: Awesome. Thank you, Sonal, for having me.

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Building a Better Chloroplast

      Judy Savitskaya and Lauren Richardson

      In this episode of the a16z bio Journal Club, bio deal team partner Judy Savitskaya and Lauren Richardson discuss research that aims to enhance the efficiency of photosynthesis and carbon fixation. These two processes are used by plants and other phototrophs (like algae) to convert light energy and carbon dioxide from the air into organic matter. The pathways took millions of years to evolve, but can scientists use advances in biochemistry and synthetic biology to increase their efficiency?

      The two articles discussed were both published in the journal Science and are both from the lab of Tobias Erb at the Max Planck Institute for Terrestrial Microbiology. The first article, published in 2016 develops a synthetic pathway for the fixation of carbon dioxide in vitro. The second article, which was published in May 2020, combines this synthetic carbon fixation pathway with the natural photosynthetic pathway isolated from spinach to create an artificial chloroplast.

      This combination of natural and synthetic components to improve the efficiency of these pathways has a number of potential applications, including in engineering our crops to grow faster. We discuss these exciting applications, how evolution has restricted the efficiency of carbon fixation and how these engineered solutions get around that problem, and the use of microfluidics for vastly improved experimental design.

      “A synthetic pathway for the fixation of carbon dioxide in vitro” in Science (November 2016), by Thomas Schwander, Lennart Schada von Borzyskowski, Simon Burgener, Niña Socorro Cortina, Tobias J. Erb

      “Light-powered CO2 fixation in a chloroplast mimic with natural and synthetic parts” in Science (May 2020), by Tarryn E. Miller, Thomas Beneyton, Thomas Schwander, Christoph Diehl, Mathias Girault, Richard McLean, Tanguy Chotel, Peter Claus, Niña Socorro Cortina, Jean-Christophe Baret, Tobias J. Erb

      a16z bio Journal Club (part of the a16z Podcast), curates and covers recent advances from the scientific literature — what papers we’re reading, and why they matter from our perspective at the intersection of biology & technology. You can find all these episodes at a16z.com/journalclub.

      Show Notes

      • Why natural processes around carbon fixation can be slow and inefficient [2:23]
      • Possibilities around improving plant performance, capturing more carbon dioxide [5:37]
      • Three key advances of this research [7:15], and the steps needed to bring it into the real world [13:08]
      • How this research could lead to the creation of artificial cells and other improvements over natural biology [14:54]

      Transcript

      Lauren: Hello, I’m Lauren Richardson and this is the “a16z Bio Journal Club.” This is our podcast where we cover recent scientific advances, why they matter, and how to take them from proof of principle to practice. In today’s episode, I’m talking with bio deal team partner, Judy Savitskaya, a resident expert in all things synthetic biology. We cover recent research that seeks to improve the processes of photosynthesis and carbon fixation, and how these advances could one day be used to improve crop growth and carbon sequestration in plants.

      First, a quick biochem refresher. During photosynthesis, also known as the light cycle, light energy is captured by chlorophyll and then passed through a series of reactions to the energy-rich chemical co-factors ATP and NADPH. These co-factors are then used by the carbon fixation cycle, or dark cycle, to drive the capture and conversion of carbon dioxide into more complex carbon molecules like glucose. Plants and other phototrophs use these two processes to turn sunlight and carbon dioxide from the air into organic matter. These are hugely powerful processes that have generated essentially all the organic matter on earth, from the wooden trees to our own bodies. But these processes also aren’t perfect, and scientists have for decades been trying to make them more efficient.

      The two articles that we discuss today were both published in the journal “Science” and are both from the lab of Tobias Erb at the Max Planck Institute for Terrestrial Microbiology. The first article, published in 2016, develops a synthetic pathway for the fixation of carbon dioxide in vitro. The second article, which was published in May, combines this synthetic carbon fixation pathway with the natural photosynthetic pathway isolated from spinach to create a synthetic chloroplast. This combination of natural and synthetic components to improve the efficiency of these pathways has a number of potential applications, including engineering our crops to grow faster. Judy and I discuss these exciting applications, how evolution has restricted the efficiency of carbon fixation (and how these bioengineered solutions get around that problem), and the use of microfluidics for vastly improved experimental design. But first, we start with a discussion of why the dark cycle, this process of carbon fixation, is not as efficient as it could be.

      Limitations of natural carbon fixation

      The key thing here is that the dark phase has this great limiting step, which is this enzyme known as RuBisCO. It is just super slow. And that’s the first enzyme in the pathway that binds carbon dioxide.

      Judy: Poor old RuBisCO — when I imagine it, it’s like an old man enzyme with, like, a long white beard and it makes a lot of mistakes and it goes really slow — but it evolved really early on, and then was a key requirement for these organisms to live. Furthermore, RuBisCO makes a lot of mistakes, which is that it often subs in oxygen molecules for carbon dioxide molecules. So there’s a huge body of work trying to evolve RuBisCO to be better, but as it stands, our plants are stuck with this really old enzyme that is not as efficient as it could be.

      Lauren: Yeah. Instead of evolving RuBisCO, it seems like plants have evolved kind of everything around it, so there’s all different classes of plants that have modified to support the slow cycling of RuBisCO, and to be efficacious in different environments and to limit the error, as you call it, of RuBisCO, which is also known as photorespiration.

      Judy: It’s kind of crazy that, like, rather than this enzyme evolving to be better, there’s entire mechanical systems involved to, like, open these pores in the plant cells to be able to let in more or less oxygen at different times of the day, and it’s this highly complex thing that has evolved to make up for the just poor efficiency of one enzyme. The Tobias Erb lab developed essentially a synthetic Calvin cycle, so it’s a different method for fixing CO2 into some sort of carbon-containing substance.

      Lauren: I say dark cycle, you say Calvin cycle.

      Judy: Fun factoid is that it’s actually the Calvin-Benson-Bassham cycle, but Bassham doesn’t want his name included because he thinks it’s a disservice to all the students that worked with him on the project, so he has requested that it be called the Calvin-Benson cycle.

      Lauren: In the 2016 article that you mentioned, the authors developed this very cool synthetic pathway for CO2 fixation that did not use RuBisCO. Instead, it used a combination of 17 different enzymes from nine different organisms that could do this dark phase half the reaction 10 times faster than the plant version that does rely on RuBisCO. And they called this the CETCH cycle or the C-E-T-C-H cycle.

      Judy: In the previous paper, they sort of cheated by adding in these enzymes that would just produce NADPH and ATP as starting points for their synthetic carbon fixation cycle so that they can kickstart part of the experiment that they really cared about. In this new paper, what they’re doing is adding in a module to create that NADPH and that ATP that is light-driven. So it doesn’t require the experimentalist to add in these enzymes or to add in the substrates for these enzymes.

      Lauren: Yeah. What they’re doing here is they’re linking the light cycle, so the photosynthetic element to the dark cycle, the carbon fixation part. So the goal is to have this own self-sustaining reaction because that’s what plants are. So let’s talk about the implications of this research.

      Real-world implications

      Judy: The biggest and most interesting implication here is that you could use some of the insights from these papers to upgrade how plants perform. And the idea is to basically counteract some of the evolutionary pressures that were present when we weren’t using these plants for crops, or to sort of make up for some of the inefficiencies of natural selection — like, for example, RuBisCO being a bad enzyme. This entirely new cycle for doing carbon fixation could really dramatically increase the rate of carbon fixation and the rate of growth for plants that we use as crops.

      Lauren: These synthetic chloroplasts that they created are actually more efficient than natural chloroplasts, and that’s because they don’t have RuBisCO, which is slow. And they also don’t suffer from photorespiration, which is that wasteful process we were talking about, where RuBisCO uses oxygen instead of carbon dioxide. And, in most plants, they waste about 25% of their energy from photosynthesis on photorespiration. So there’s this way in which you could kind of get around the photorespiration problem with something like these synthetic chloroplasts.

      Judy: When we think about, on a global scale, the carbon cycle, and if we’re concerned about release of too much carbon into the atmosphere, there’s sort of an interesting class of solutions here, which is to increase the rate at which our crops pull carbon dioxide out of the atmosphere, and that kills two birds with one stone. One is that it increases your efficiency of food production, and at the same time, you’re removing more carbon dioxide from the air. You’re actually using it for something useful.

      Three key advances

      Lauren: Yeah, that’s possibly a very elegant solution. Let’s dig into these methods and results now. So in plants, photosynthesis happens in chloroplasts, and chloroplasts contain an internal membrane structure called thylakoid membranes which contain chlorophyll, the molecule that actually is able to capture light energy and convert it into energy that the plant can use. And all the other enzymes in the pathway that are needed to go from light energy to ATP and NADPH, which are these energy-storing molecules.

      So the way I see it, there were three key advances in this paper. The first was extracting these membranes from spinach that contained the enzymes for the light cycle, and getting that into a functional unit; then linking it to this synthetic CETCH cycle — this synthetic carbon fixation pathway that they’ve created — and then the third was to use microfluidics to really optimize and integrate these two cycles together so that there was this self-sustaining basically synthetic chloroplast.

      Judy: I mean, I think it’s cool that they’re able to show you can get this thylakoid membrane module, separate it from the rest of a chloroplast, which is integrated complex, large organelle. They can just take this one piece of it, and then it works like the black box you would expect it to. There was one change they had to make, which was to add exogenous ferredoxin, which is like the one component of this, sort of, electron transfer process that is not attached to the thylakoid membrane. Other than that, it kind of just transferred wholesale into this in vitro context and worked. So I’m sure there’s, like, lots of experiments here that were failures that we’re not seeing, or that are, like, buried in the very, very large supplemental materials for this paper, but it’s really impressive that they were able to basically show the function of this module in vitro without all the bells and whistles surrounding it from the natural organism.

      Lauren: So next, they linked these thylakoid membranes, the part that’s performing photosynthesis, to the synthetic CETCH cycle. What do you think about this fusion of the natural and synthetic components? Because that’s what they’re — basically they’re doing here. They’ve got the natural photosynthesis machinery, and then they’ve got the synthetic dark cycle machinery.

      Judy: Yeah, it’s interesting because it’s sort of, like, demonstrating that we understand half of it, right? So there’s this — there’s two approaches to understanding the parts of a system. There’s the bottom-up and top-down. So if you understand all of the components of some enzymatic pathway, you should be able to add them all in, one at a time, purified, and then recapitulate the behavior of the full pathway. So that’s sort of what they’ve done with their first paper with the CETCH cycle, and then there’s a different way that biologists understand nature, which is by breaking it down. So you start with, like, this is how the organism works — and then take away pieces until you figure out what’s like the set of things that is necessary to do a certain reaction. And this is kind of cool because it’s a fusion of both of those worlds.

      Lauren: Yeah. I think there’s something interesting, and the rate-limiting step is this RuBisCO, that’s part of the dark phase. It makes sense to tinker with that element, but you don’t have to reinvent the photosynthesis arm, the part that is working. You can appreciate, kind of, the beauty that nature has already provided and use that in combination with the things you want to change.

      Judy: Yeah. That’s a really good point, actually. I hadn’t thought of that, but this really suggests that you can move this CETCH cycle that they’ve engineered into an organism that already has that thylakoid membrane piece intact, and you should expect them to just work together well.

      Lauren: So, and the third aspect of the paper, they’re using microfluidics to integrate the thylakoid membranes with the CETCH cycle, and to create these basically artificial chloroplasts. So talk to me about what they did with the microfluidics, and what the benefits of using microfluidics for this approach are.

      Judy: Yeah. The real benefits of droplet-based experiments is that you can do many of them at once. So the idea here was to create lots of these little droplets, so that each one can contain a different experiment with a slightly different version of the CETCH cycle, or a different ratio of these components that they’re putting together. And they used color-based barcoding, so they could tell what reaction was happening in a given droplet by changing the amount of these different dyes that they added in. The idea is to basically be able to do many experiments in parallel and look at them in one go.

      Lauren: So basically, it’s a way to multiplex the experimental design.

      Judy: Yeah, that’s — that’s a perfect way to say it. There was this interesting figure at the end where they showed that they get more production of glycolate, so sort of, like, output of their process in the droplets than they do in bulk solution, given the same amount of chlorophyll to start with.

      Lauren: My understanding was that it’s all about the right amount of cofactor regeneration, so ATP and NADPH regeneration from the thylakoid membranes to support the optimal functioning of the CETCH cycle. And then, do you think the inherent next step is using microfluidics? Would they be able to, kind of, dose in the exact amount for optimal production?

      Judy: Yeah. I mean, they’ve got 17 enzymes to play with, so that’s, like, a lot of parameters that you can modify, and then you can change the levels of each of those enzymes. So this microfluidic tool gives them the opportunity to test, like, at very high levels of multiplexing how to optimize this cycle and optimize its interaction with the thylakoid membrane.

      Introducing new processes to plants

      Lauren: I’m wondering how many steps do you think there are between this work and, like, what they’ve achieved now, and actually getting that into plants?

      Judy: That’s actually a really interesting question, because they’ve shown that this synthetic, like, hodgepodge enzyme set works in vitro. That does not mean that it’s going to work in vivo at all. So the first thing is to put this into some really simple organism that’s easy to engineer, like an algae. And the idea here is that you would use the natural thylakoid membranes activity from that organism, but then it would express the enzymes from this different CETCH cycle instead of the natural Calvin cycle, and what you’d need to do is a ton of optimization. I’m not going to sugarcoat it. So is this on the horizon? Probably not. I think the microfluidic experiments that they have are going to be helpful because if they can start with, sort of, extracts of this algae, put it into these microfluidic experiments, and then do their multiplexing there, they can do many more experiments at once, but there is still going to be a big jump from that to the actual organism.

      Lauren: Yeah. It’s kind of like the benefit of the CETCH cycle was that they could use all of these different enzymes from all of these different organisms and create this brand new pathway, which was so neat in vitro, but that creates a whole host of new problems for that in vitro to in vivo switch.

      Judy: Yeah, absolutely. I mean, I think that’s actually where a lot of, like, the interesting insights into biology come from is, like, we understand how the system works in isolation. We put it into the context of the cell and suddenly everything breaks, and so now the question is like, why did it break? So lots of cool biology coming from trying to transfer this work, but I would not expect, you know, next year to see a paper where this cycle is fully functioning.

      Possible future applications

      Lauren: The authors of this paper really blew my mind with the last paragraph of their discussion where they talk about using these synthetic chloroplasts in combination with other life characteristics such as self-repair and reproduction in the idea of basically creating a fully artificial cell. When you start thinking about fully artificial or synthetic cells, you know, that makes you think about fully artificial or synthetic tissues, and that kind of scales up to a fully synthetic organism — having the ability to synthetically harness the light from the sun, carbon dioxide from the air, and turn this into, you know, a designer metabolic pathway that could fuel a synthetic life force — is very exciting to me and just kind of wild to think about.

      Judy: I love the term synthetic life force. If you think about the cell and all of its functions as a graph, like in the classical computer science sense of the word graph, it is like a super complex structure with, like, many interacting nodes and it’s, like, very hard to get your head around it. How could you ever build that from scratch and make it self-sustaining? But, like, this is a really big piece of that — generating energy, making it happen without an external agent putting in that — putting in new molecules. Like, that could — that could handle, like, a very large portion of the graph that is necessary to make life work. I will say what this gets you is that you don’t have to feed sugar, right?

      Lauren: There’s definitely something about the, like, independence of it, though. Like, there’s, you know, providing sugar or feeding it, versus being able to create those energy-storing molecules de novo, which can then be turned into mass, or broken down again as sources of energy. I think it’s really interesting to think about, like, what are the essential processes that you would need to create a fully self-sustaining, independent-of-human-support system that is lifelike in this way, you know, based on biology and not, you know, a robot that we build in the lab.

      Judy: But also, like, how do we define lifelike? Just because we metabolize the particular chemistry that we use to do that, that’s just one instantiation. It’s kind of like what happened to result from evolution and then, like, stick because it’s really hard to evolve out of this, maybe even local minimum, maybe not global minimum in terms of how good the processes are. So yeah, I think let’s definitely push on synthetic cells. I think it makes a lot of sense to start with, like, things that look like existing biology, but, like, why stop there? Why not go to something that’s sort of hybrid or exploits entirely new chemistry that we’ve never seen?

      Lauren: Yeah. And this kind of can even get us back to what we were talking about at the beginning which was, like, how bad RuBisCO is as an enzyme. RuBisCO originally evolved in environments where there was not a lot of oxygen, so it was before the great oxygenation of the atmosphere. And so, this problem with substituting oxygen for carbon dioxide just wasn’t a thing when it first evolved. And as oxygen increased in the atmosphere, it had to start making trade-offs between the specificity of whether it chose oxygen or carbon dioxide, and its efficiency — so it could be more efficient, but then it would incorporate oxygen more often, versus it could be more specific but then it would be even slower. So if you’re designing a system de novo, is there a way to bypass some of the evolutionarily inherited tradeoffs and make something that’s just more finely tuned to the situation that you want to design?

      Judy: So evolution is kind of always lagging behind how the world is changing, which is exactly why RuBisCO is evolved for a world that we no longer live in — but humans can adapt much faster. That’s, like, this interesting philosophical idea that people will say evolution has infinite creativity — like we could never, you know, think up the things that evolution has created. And I think that’s true to some extent, but evolution is fundamentally limited to the designs that are within a certain distance of the designs that are out there in nature today. You’re not going to get a really huge rapid change in an organism just because it wouldn’t survive that sort of transition period. So there’s all of these transitions that evolution can’t pass through, but we can as humans. So I actually think, like, in a lot of ways, human creativity can go way beyond what evolution has made, and I think there’s, like, a ton of opportunity here.

      Lauren: Yeah. I don’t think it’s necessarily about being better than evolution. It’s learning from evolution and seeing all the different ways that evolution has functioned and then kind of taking, you know, the best of the best <Plan matching…> and our own — yeah, our own knowledge. And, you know, what AI will be able to provide to us is, like, even beyond our own knowledge is, like, new ways of looking at these problems and these solutions and, like, being able to input them in completely creative ways that, you know, evolution hasn’t found yet and neither have people.

      Thank you, Judy, for joining me on “Journal Club” this week. To sum up, we are excited about this work, as it demonstrates that you can improve the process of carbon fixation and link it to the natural photosynthesis machinery from plants. This bioengineering solution could be applied to our crops to improve growth efficiency and carbon dioxide sequestration. That’s it for “Journal Club” this week. You can find all these episodes at a16z.com/journalclub. Thanks for listening.

      • Judy Savitskaya is a deal partner at a16z where she focuses on bio companies. Previously, she worked on synthetic biology research at UC Berkeley and was a computational modeling and neuronal networks researcher.

      • Lauren Richardson

      Section 230, Content Moderation, Free Speech, the Internet

      Mike Masnick and Sonal Chokshi

      In this special “2x” episode (#32) of our news show 16 Minutes — where we quickly cover the headlines and tech trends, offering analysis, frameworks, explainers, and more — we cover the tricky but important topic of Section 230 of the Communications Decency Act. The 1996 law has been in the headlines a lot recently, in the context of Twitter, the president’s tweets, and an executive order put out by the White House just this week on quote- “preventing online censorship”. All of this is playing out against the broader, more profound cultural context and events around the death of George Floyd in Minnesota and beyond, and ongoing old-new debates around content moderation on social media.

      To make sense of only the technology and policy aspects of Section 230 specifically — and where the First Amendment, content moderation, and more come in — a16z host Sonal Chokshi brings on our first-ever outside guest for 16 Minutes, Mike Masnick, founder of the digital-native policy think tank Copia Institute and editor of the longtime news & analysis site Techdirt.com (which also features an online symposium for experts discussing difficult policy topics). Masnick has written extensively about these topics — not just recently but for years — along with others in media recently attempting to explain what’s going on and dissect what the executive order purports to do (some are even tracking different versions as well).

      So what’s hype/ what’s real — given this show’s throughline! — around what CDA 230 precisely does and doesn’t do, the role of agencies like the FCC, and more? What are the nuances and exceptions, and how do we tease apart the most common (yet incorrect) rhetorical arguments such as “platform vs. publisher”, “like a utility/ phone company”, “public forum/square” and so on? Finally: how does and doesn’t Section 230 connect to the First Amendment when it comes to companies vs. governments; what does “good faith” really mean and what are possible paths and ways forward among the divisive debates around content moderation? All this and more in this 2x+ long explainer episode of 16 Minutes.

      Show Notes

      • An explanation of what Section 230 is and what it covers [2:03]
      • Publishers vs. platforms and a discussion of current events [6:37]
      • Why platforms are not legally considered public utilities or “public squares” [11:57]
      • An overview of the Executive Order on Section 230, and the powers of the FCC [18:10]
      • How the Executive Order restricts federal spending on platforms [27:13]
      • The difficulty of content moderation and why Section 230 protects all websites [30:23]

      Transcript

      [updated intro as of January 8, 2021 here]

      What is Section 230?

      Mike: The law is actually very short, and very simple, and very straightforward. And I should note, that the Communications Decency Act itself did have many more things that it did, but all of that was determined to be unconstitutional. So the only thing that survives is Section 230. There was a big lawsuit, ACLU vs. Reno, in the late 90s, and that threw out most of the Communications Decency Act as unconstitutional; the thing that remained was 230.

      So Section 230…really does two things, and they’re somewhat related, and they’re both incredibly important to the functioning of the modern internet. The first thing that it does is it puts the liability on the person actually violating the law. So, if someone goes onto a website, and says something that is defamatory or other otherwise violates the law, the liability for that action belongs on the person who is speaking — and not the platform or site that is hosting that content.

      The second thing that it does is that if a website chooses to moderate its content (or anything that is put on the site), then it is not liable for those moderation choices.

      Sonal: I’m so glad you’re bringing that up because this is the #1 thing I wanted to start with, which is, the flip side of it — not just the protection, but the fact that they can moderate whatever they want — so can you actually break that down, Mike? What does that mean?

      Mike: So, where it came from — which I think is important to give sort of the history very quickly — is that, there were a series of lawsuits in the early 90s that tried to hold internet services that had moderated some content. There were defamation cases, effectively, brought up.

      The most famous one is Stratton-Oakmont vs. Prodigy and as a little fun aside, Stratton- Oakmont was a financial firm that was immortalized in the movie “The Wolf of Wall Street” —

      Sonal: That’s a fun fact.

      Mike: Yes. And Stratton-Oakmont got upset because people in Prodigy’s message boards were accusing the company of being up to no good and so they sued Prodigy.

      A court said that Prodigy was liable for the libelous statements because Prodigy positioned itself as a family-friendly service that would moderate content. Because it moderated some content — [i.e. taking] down cursing or porn (or anything that it felt was inappropriate) — anything left up (according to the judge) [Prodigy] was now liable for as if it had written that content.

      And that freaked out people in Congress namely…two members of the House: Chris Cox (a Republican) and Ron Wyden (a Democrat). They put together Section 230 to say wait, that’s crazy. If a website wants to moderate content to create, for example, a family-friendly environment, it shouldn’t get sued for the content that it chose not to take down.

      And so that section of CDA 230 is designed to make sure that any website can moderate content how it sees fit, in good faith, to present the content in a way that meets with the goals of the service.

      Sonal: Right. And to be clear, these are not just quote “content moderation things.” It could be spammy posts [or] the kind of thing that would actually turn you off from using the service [i.e.] family-friendly site getting rid of porn. The companies can use whatever discretion they wanted, as long as it complies with their terms of services, which could change.

      But what’s interesting about this back-story: It’s a very small thing that was preserved, but that had huge consequences for where we are today, in terms of the internet we have today. Whether it’s going on a recipe-swap site; whether it’s sharing photos of family and friends; whether it’s posting a car for sale — there are so many layers to this. It has allowed the modern internet to thrive. One of the best lines I heard (I think this was actually in Verge) is that, in many ways, this Act was a gift not to big companies, but a gift to the internet.

      Mike: I think the point is not that it is the biggest gift to big internet companies OR that it’s the biggest gift to the internet — I think it’s really the biggest gift [of] free speech for everybody, right? Because if you don’t have 230 set up the way it is set up, there would be much more limited ability for users to actually post content online.

      It’s a little bit crazy to me that people think that changing or getting rid of 230 will enable more free speech, when the balances that are set up within 230 are very much designed as a gift to free speech.

      Platforms vs. publishers

      Sonal: Okay, so now my question for you is, given that we did enter this world where user-generated content — whether on sites like YouTube with videos, educational or non-educational, political or non-political — we now live in a world where…people often use the framing of “platform versus publisher” (which I think is kind of meaningless and arbitrary). Sometimes, [they also] use the ridiculous phrase, “platisher,” as a hybrid of the two. I’d love to get your take on that framing and how that doesn’t (or does) apply here?

      Mike: So, one of the things that comes up over and over that people say is, “Well if they moderate or if they change content, they are no longer a publisher; they are now a platform, and therefore, they lose Section 230 protections.”

      The law makes NO distinction between platform and publisher; the law is not designed to protect one or the other, or say that there is difference between it — there’s no classification. It’s not a safe harbor where you have to meet, you know a, b, and c criteria in order to get the protections. You just need to be an “interactive computer service” that hosts third party content.

      So the debate over “Are they a publisher, or are they a platform?” is completely meaningless under the law.

      Sonal: Let’s actually talk about some recent events because I think it’s a useful case in sort of understanding 230 — and then we can break down some of the recent news as well around that.

      So one recent event is that Twitter added a feature earlier this week where, [on] one of the President’s tweets, they added a link to other sites as a sort of quote “fact-check” mechanism. This could be contentious because a lot of people do not actually believe that everything the media writes is correct. That said, it linked to other third-party news sites, and it kind of labeled it as a fact-check feature.

      Then they added another thing where they kept a tweet from the president up — in the context of the Minnesota George Floyd protests — but put like a limit on it where people could retweet with comment but they couldn’t retweet, like, or reply to it, because it violated their site’s terms of services around speech that incites violence.

      And so, in case one, they were adding what they quote called a “fact check” layer; in case two, they were adhering to their own terms of service around spreading violent speech, which said they kept up in the public interest.

      So that’s a super, super high-level summary of what happened so far. My question for you now, Mike, is how [Section 230] does and doesn’t apply here? Because in this case, “the fact check” could be construed as commentary content, not just third-party content.

      Mike: It’s a really complex topic. Each layer of it adds new complexities and each of those complexities are in some way important.

      Let’s do the two tweets separately. The first tweet, they added something — and just as a minor correction (and this has been going around a lot; people said this is the first time that Twitter had used this) — Twitter has been using that feature over the last couple months but, this is the first time they have done it on a politician’s tweet.

      What’s amusing to me is the time I saw it used…about two weeks earlier. It was used to debunk a Jimmy Kimmel video that was making fun of Mike Pence. Twitter put on a thing that said this is manipulated media, it is not accurate. It was a tweet that had gone viral. It was making fun of something that Pence had done, and Twitter stepped in and said no, this is incorrect or manipulated media. [Twitter then] had a link to third party content saying why it was manipulated.

      And so, that is allowed under 230. What it is doing is adding more speech — it is linking to other sources [and] providing more context. The part that is not protected by 230, was never protected by 230, and no change to 230 is going to change that, is any speech that comes directly from Twitter itself. So, in this case, that was the very narrow line that was put under Trump’s tweet. It said something like, “Get more facts about mail-in ballots,” or something to that effect. That particular line is from Twitter itself and therefore is not protected by 230. But, it IS completely protected by the First Amendment. The third-party content that they link to is then protected by 230.

      [In] the second tweet, Twitter did something new (which I had not seen before) in which they put up a note that said that this tweet violates the terms of service; however, they want to keep it up because they feel that [the tweet] is relevant and important for people to see [its] content but to understand that it violated the terms of service. So they added more context and limited the ability for people to retweet or reply to it.

      And again, this is 100% allowed by 230. It did not remove the content, it didn’t take it down. Even if Twitter chose to take it down, or take down his speech, or take down that tweet, that wouldn’t violate his free speech rights. The First Amendment protects people from the government acting, not from a company. Now, I have also since seen Twitter use “This tweet violates our terms of service, but we are leaving it up because it is newsworthy,” message on at least one other tweet this morning from somebody who was defending the president.

      Platforms as a public square

      Sonal: Let me ask you another question, and then we can break down the executive order…Since we already debunked this platform-publisher distinction, [what do you make of] these companies that provide these interactive web services are like phone companies? They always use this line that, “Oh but imagine if the phone company decided to take down that conversation you had and interrupted you in the middle?” What do you make of that analogy?

      Mike: Yeah so that is popular in a wide variety of circles across the political spectrum and doesn’t fall into any sort of partisan viewpoint — sort of the “public utility” argument.

      Sonal: And that by the way, of course reminds me of Net Neutrality, which we both covered quite a bit.

      Mike: Right. There are some funny parallels between this situation and Net Neutrality in that a lot of people’s positions are reversed from one to the other.

      Sonal: We don’t have the time to talk about Net Neutrality, but I covered it extensively at WIRED, as you know, and from all different perspectives: from carriers, to FCC, to internet companies, you name it. That is exactly what’s fascinating to me, is that the positions and the sides are inverted in this case. So, anyway, what do you make of the phone and common carrier-type argument?

      Mike: So it’s an important one to understand, but I don’t think it applies. I think that most people who are deeply familiar with public utilities and what is required to be declared a public utility would recognize that internet services — what’s sometimes called “edge providers” (which are the services that you and I use every day, that we interact with) — do not qualify and do not meet the requirements of a typical traditional utility service. To clarify, what that means [is that] usually a utility service is something that is offered to everybody, but is also commodified. If you use AT&T, or Sprint, or Verizon, you are getting the exact same service. There is no real differentiation in terms of the service that you’re getting — it is a commodity, one provider to the other. Same thing.

      That is not the case with various internet edge providers i.e. Google, YouTube, Twitter, Facebook. Each of them have all of these different features and all these different things. They are not 1:1 replaceable. It is not a commodity that you can switch out; therefore, the public utility argument does not really apply.

      You can argue that there should be some other kind of classification (and some people do argue that), but comparing them directly to a telephone service is different because it’s not core infrastructure — [they’re] things that are at the edge, things that you use as a service provided beyond that.

      Sonal: What do you then make of this “public square / public forum” argument?

      Mike: People say that Twitter or Facebook shouldn’t be allowed to do any moderation or take down any content because it’s the new public square, and therefore violates their rights. They will often point to two different lawsuits in making this argument: one is Pruneyard, the other is Packingham. These two cases… have been brought up in a whole bunch of lawsuits and I’ll just say that, every time they’ve been brought up in a lawsuit to argue that a social media site is the public square, they have failed. I have not seen a single judge anywhere agree that these things make any sense in this context.

      But just to give the quick background on the two cases, and they go deep, but I’m going to give as high-level as I can, and as quick as I can: Pruneyard was a case about a mall that was trying to kick people out, effectively. It was argued that the mall was a gathering place and became the sort of de facto public square and that took away some of the rights of the private property owner [of the] mall to kick people out. The court said that it was a de facto public square and they could not kick people out.

      Now it is an extraordinarily limited ruling and extraordinarily focused on the facts of that case. [The mall] was effectively the only place in town that anyone could gather. The mall owner sort of acted as a local government and was therefore replacing government functions — functions that normally were done exclusively by the government. Every other case after that that references Pruneyard has effectively limited it; it only applies in a very, very narrow situation, which is basically Pruneyard and Pruneyard alone. You can’t just say that something is a public square.

      The Packingham case is a more recent case. It was a Supreme Court case that kicked out a state law that basically said if criminals had done some sort of criminal activity online, part of their punishment could be that they are barred from using the internet. The Supreme Court said you cannot pass a law that kicks people off of the internet because the internet is so essential to people’s lives and ability to work and all that kind of stuff. So people have taken that to mean [that] the services themselves cannot kick people off, but that is not what the case has said — it just said that the government cannot pass a law that forces people offline.

      There is a third case that people never mention but is the most important case. It was just decided last summer, and that is the Manhattan News Network case. I won’t get into the details but the Supreme Court ruling was written by Brett Kavanaugh, who was the most recent appointment, and his ruling said that you can’t just declare any place where people can speak — even if a lot of people speak there — a public square. It doesn’t become a state action; it doesn’t take on government control.

      The idea that something is a public square or that there is state action involved from a private company only applies in a very limited set of circumstances where that service or operation is, again, replacing activities that were traditionally done by the government. The ruling makes it very clear that Twitter, Facebook, YouTube, and every other website out there does not qualify. They are not replacing government [and] are not offering services that were traditionally given by the government.

      Sonal: Right. This basically means that if the sites DO perform services that are exclusively a service provided by the government (i.e., if the government decided that all tax reimbursement would be done entirely online and no longer through the U.S. Postal Service), they would then have to comply [with] provisions.

      Mike: Right, there could be an example, something that was traditionally and exclusively handled by the government. I could see an argument where someone could not be kicked off or blocked because that would imply state action issues.

      Overview of the Executive Order

      Sonal: So now let’s talk about the news — again, as a way to explain what CDA 230 is and isn’t. We’ve explained and debunked some of the myths and framings around arguments of platforms vs publishers and analogies to phone systems. Let’s talk very briefly about the recent executive order that was issued this week…[and discuss] what the executive order can and can’t do here, or what it purports to do and doesn’t do.

      Mike: There were drafts of this executive order that made the rounds over the last two years. This is something that the White House has been thinking about. I reported on it, a number of other news sites reported on it as different drafts were leaked out to the press about earlier versions of this executive order. And, the story is that, in the past, they’ve passed this around to different agencies like the FCC and the FTC. The message that the White House got back was that this was unconstitutional, and they couldn’t do any of this, but it seemed that they took it out of the drawer, dusted it off, and then put a fresh coat of paint on it.

      [The executive order] says a lot of very angry stuff about the internet services and platforms and the way they handle moderation. There are seven different sections but [there are] two sort of scary-ish parts of the executive order; to one extent, the order effectively tasks the FCC with coming up with a new interpretation of 230 [but] hints very strongly what the FCC’s interpretation should be — that interpretation is totally at odds with both what is written in the law and what 20 years of case law have said.

      That’s worrisome only to the extent that anyone would ever actually pay attention to that FCC interpretation. The FCC in ACLU vs. Reno (which is the lawsuit that rejected and made most of the Communications Decency Act unconstitutional) made it extremely clear that the FCC has no authority whatsoever to regulate websites. None, zero, zilch. It’s not even an open question — they cannot.

      Sonal: And just to be very clear here, the FCC (Federal Communications Commission) is an independent agency; it has a five-member commission. I believe there’s currently three Republicans, two Democrats.

      Because it comes up a lot what FCC can do / can’t do, like, it cannot make laws, but it does have the ability to interpret existing laws and put out certain rule making things. They do these Requests for Comments which create public records of people’s commentary and whatnot. They also have the power to ask for documents — and they can do distracting things — but they may not have legal-making authority. So I think it’d be very helpful for you to break down a bit more specifically what they can get away with and also can’t.

      Mike:  So they can do rule making, and that is a long involved process. Interestingly, because [the FCC] is an independent agency, the President cannot instruct them to do something. The executive order instructs the NTIA (which is part of the Commerce Department) to ask the FCC to do this. Technically, the FCC does not need to do this, but the FCC will certainly feel the pressure to probably do something. The FCC could certainly create a lot of a lot of nuisance, and yes, there will be comment periods, and people’ll have to testify, and put in comments. As we saw with the Net Neutrality hearing, the comment system was filled up with bots and nonsense, so the commenting and the rule making process is a bit fraught with distraction.

      And so yes, [the FCC] can make rule making and can do something to enforce that rule making. If the rule making covers things that it is authorized, that the FCC is authorized to have regulatory power over by Congress —

      Sonal: That are in its jurisdiction, so to speak.

      Mike: That are in its jurisdiction — and websites are clearly not. Congress has never said that websites are within the FCC jurisdiction, and the main court case that tested the theory that websites were in the FCC jurisdiction has said no.

      One other thing that I do want to note about the executive order and the request to the FCC is that, it is couched in a term that totally misinterprets CDA 230.

      Sonal: Which is?

      Mike: So earlier, I talked about the two different parts of the CDA — that one is about liability on third-party content, and one is about the platform’s protection in moderation. There are a few very narrow conditions on that moderation ability. They say: it has to be in good faith; and there’s a list of different content that you can moderate that includes otherwise objectionable content.

      That otherwise objectionable content is very, very broad — it can cover basically whatever the platform thinks is otherwise objectionable. In order to argue good faith, [you] would open up a whole other First Amendment can of worms.

      But what the instructions to the FCC indicate is that those limitations — the good faith, otherwise objectionable stuff — somehow applies to the first part of CDA 230, which is the part about not being responsible for third-party content. That has never been the case. Nobody’s ever suggested it is the case. It has never shown up in any lawsuit. It has never been argued in a legitimate way and yet, the executive order suggests that the FCC should look into whether or not that interpretation makes sense.

      Sonal: So you’re basically saying that the two provisions of CDA 230 — that people are not liable for libelous content that their users might put on their site (or any other content their users might put on their sites) — is being conflated in this case with the good-faith aspect of being able to discretionarily moderate in “good faith.”

      Mike: Exactly; they’re sort of mixing those two things up. I would argue that is done in bad faith, to make use of the good faith, limitation on all this.

      Sonal: Oh, man. Right. What other aspects of the executive order — again, without going into breaking down every little detail because this is really more about the underlying principles — would you say have impact for understanding and really interpreting and explaining what CDA 230 is and isn’t?

      Mike: So one important part — and this was added at the last minute, perhaps literally, because the draft that was leaked the night before did not have this, but the final executive order did have it — is that, it instructs the Attorney General to draft a law — oddly, not a federal law, but to draft like a reference state law — to effectively reinterpret CDA 230 in a way that diminishes its power.

      And that could be problematic. Here’s an aside that I probably should have brought up earlier: 230 is not a universal immunity. It is not as universal as people make it out to be. One thing that it does not cover is federal criminal liability. So, if you break a federal law — drug trafficking, human trafficking —

      Sonal: Child pornography, etc.

      Mike: Child pornography, all of that stuff — the sites are still liable; 230 specifically exempts that. So, the Justice Department and the FBI, if they felt that any of these platforms were violating federal law, they have always, always under 230 been able to go after those sites and that includes third-party content. There are a whole bunch of conditions on that.

      So if there is drug dealing, human trafficking going on those sites, [they] potentially could be criminally liable. The Attorney General, and the Justice Department, and the FBI have always had only way to make use of the law to go after these sites. And yet for the last few months, the Attorney General has been attacking 230 and acting as if it limited his power in some way when it simply does not. But now he can draft a law, and, he’s sort of already been doing that.

      Sonal: What’s so amazing about what you just said, though, Mike, the part about the federal part actually immediately reminded me of the encryption debate — which we actually have discussed on this very show “16 Minutes” (and listeners can listen to our reframing of that debate); another place where policymakers on both sides have very conflicted views on.

      Mike: Yeah, and there’s already a bill that’s in Congress — that was put together with the help of the Attorney General — and it sort of ties the 230 debate to the encryption debate. And, it’s very convoluted.

      Sonal: Oh, this is the EARN IT–

      Mike: The EARN IT Act. And what it has the potential to do is to say that if you are offering end-to-end encryption on your service, you no longer get 230 protections (it’s a little more complicated than that); but, his abilities to do that in a manner that would remain constitutional is a pretty big question.

      But again, it could create a huge nuisance. And part of this is also [the Attorney General is] going to establish a working group so there will be discussions, roundtables, panels, hearings, subpoenas and all sorts of things that are going to happen in the meantime that are designed to be an intimidation tactic to try and — the phrase that everyone uses is “work the refs”, right? Tt means basically, “Hey, Twitter/ Facebook/ YouTube, if you don’t want us to keep causing trouble for you, maybe don’t be mean to us,” you know. Don’t fact check us. Don’t limit our tweets. Don’t limit our content. Don’t put extra notices on it or other limitations on it because the more you do that, the more of a pain we’re going to be to you.

      Restrictions on federal ad spending

      Sonal: To summarize, at a super high level, the FCC has extremely limited jurisdiction over websites, specifically; the Attorney General does have some ability.

      We haven’t talked about what’s not in the executive order, but this is where there’s a little bit of the dust storm [that] is very distracting which is that Congress could choose to rewrite policy (if they wanted) using this as an incitement for that.

      Mike: Yeah. There are people in both the House and Senate who have said that they will introduce legislation based on this and try and do more than the executive order can do. Whether or not that legislation can actually go anywhere, any such legislation would almost certainly be subject immediately to a First Amendment challenge and would likely fail, but that would be many years into the future.

      Sonal: Right. So we forgot one bit of the executive order, which is probably the only legit thing in there seemingly, which is that part of this had the threat of limiting any government dollars of advertising going to these sites. And I by the way did a little quick check and based on federal procurement records (this is according to The Verge) apparently, only $200,000 of advertising have been provided to Twitter specifically since 2008 — which sounds a little crazy to me. It can’t be getting everything, that seems way too low but even still, it does suggest that the government advertising is actually a very tiny piece of the bottom-line revenues of these companies. But, I’m curious for your take on that.

      Mike: Yeah. So, that is one thing that an executive order actually can do, right, which is instruct certain federal agencies in terms of how they’re spending their money in some form or another —

      Sonal: — Oh by the way, to be clear, when you say “their money,” we’re actually still talking about taxpayer money here.

      Mike: Yes, yes — mostly taxpayer money. There are a few exceptions, but mostly taxpayer money is what we’re talking about here. What’s funny is the executive order sort of implies that it is telling agencies to stop spending on these websites, but it doesn’t actually say that. It says they have to account for what they’re spending, and they have to submit it to the Office of Management and Budget, and then something maaay happen in the future based on that. And the implication is that they should not be spending.

      So, there could be a tiny, tiny, tiny miniscule drop in spending and, what’s silly of course is that, I would bet that the various political campaigns of everyone who is cheering this on are still spending much more money themselves as campaigns on these social media platforms in order to advertise.

      Sonal: No question! On all sides.

      Mike: So the one concern from a societal perspective is that the few federal agencies that do advertise on social media, actually probably have pretty good reason for that, and the one big example is the Census Bureau. And it’s 2020, and we’re in the midst of supposedly collecting the census.

      Sonal: I forgot about that, right.

      Mike: Because every 10 years, we have to do a census.

      And one of the best ways that the government has found to get out the word, and to get people to actually fill out their census forms is through advertising on social media; therefore, pulling that budget and telling the Census Bureau that they cannot advertise actually could limit the ability of the Census Bureau to collect the data that they are required under the Constitution to collect.

      The difficulty of content moderation

      Sonal: So, Mike, this is a wonderful summary so far of what Section 230 of the Communications Decency Act does and doesn’t allow; of the recent news, what’s hype/ what’s real, and sort of really using that to explain these laws that have allowed our modern internet. I will be linking — just in the show notes so people know — to a lot of the articles that did good explainers, a lot of your wonderful pieces in particular, as well as the actual executive order, and the analysis of the differences that Eric Goldman (our mutual friend) put up.

      One question I do have for you — this is very much playing out against a broader backdrop of debates around big tech, debates around content moderation — is, given that the recent example did not necessarily remove or necessarily even fully restrict (except maybe in spread and engagement and scale), there [have] been a lot of complaints about things like shadow banning. There’s also a lot of conflation between content and behaviors (like what sites can do versus what they say) and for me, it seems like when it comes to this content moderation debate, you’re damned if you do and you’re damned if you don’t.

      I’m curious for your thoughts on a) where this fits in that longer/broader scape of that debate; and then b) is there a way forward in your mind?

      Mike: So I put a joke on Techdirt a few months ago and I keep referring to it over and over again. There’s a famous economist, Kenneth Arrow, he had this thing called the Arrow impossibility theorem. He looked at all different kinds of voting systems and argued that none of them can accurately reflect the will of the populace. And so I did a play on that, which I called — humbly — the Masnick Impossibility Theorem.

      Sonal: You are a very humble guy. We go way back, I think it’s been quite a number of years I’ve known you.

      Mike: I don’t even remember how long ago that was, but it was way back.

      Sonal: It might be like 15. No, not 15. Maybe 15, almost like 12 years now. I don’t know.

      Mike: Could be, yeah.

      Sonal: I love that you named it after yourself; I want to hear about the Masnick Impossibility Theorem!

      Mike: It is impossible to do content moderation well, and there are a variety of reasons for that:

      • One being that, any kind of content moderation is going to piss off someone, and that is generally the person whose content was moderated.
      • The second element is that, you know, so much of this is subjective decision making, and everybody has a different view on these things… We ran a sort of conference event a few years ago, where we made everyone in the audience have to be content moderators for a number of different case studies effectively; and we had 100 content moderator experts in the audience, and none of them agreed. On every case that we did, people had strong disagreements over what should have been done about this particular content.
      • And then on top of that, you just have the law of large numbers. If you’re making decisions on 500 million pieces of content a day, and you get it 99.999% correct, you’re still going to have a huge number of mistakes, however you define “mistakes.” You know, there are things that are going to be missed; there are things that are going to be taken down that probably should not have been taken down. That is going to happen, there is no way to avoid that. And in absolute numbers, because the overall set is SO large, it’s going to appear like these companies are incompetent in how they moderate content.

      That is just the reality of the process of moderating content, and nothing is going to fix that. Hiring more human moderators is not going to fix that; building better AI is not going to fix that. You can improve on it but one of the nice things about Section 230 — and the way it is structured in that there is no liability for the moderation — is that it allows for different experimentation to happen.

      So you have very different approaches. And, everybody focuses on Twitter, and Facebook, and YouTube — but then you have to take into account tons of other sites, including Wikipedia. Wikipedia is allowed to have all these individuals editing their platform because of 230. Or you look at another site like Reddit, right; Reddit has set up all these different subreddits, and each of them have their own moderators that allow them to set up their own rules. That’s allowed — that is possible — because of Section 230. And any of these changes could make those kinds of things impossible.

      Sonal: It’s funny because in the examples you listed, you made sites that are very often used by students, like Wikipedia for research; but also, I just wanna make the point on this, that it applies to vaccine sites, and anti-vaxxer sites. It applies to all kinds of sites and that variety is partly the point here as well. I think that’s really important to underscore.

      Mike: And let me underscore it even further. CDA 230 protects every website online. People say that, “Oh it’s a gift to big tech and newspapers don’t get this.” No, newspapers get it too for their website; every website gets this, and that means your personal blog. It means when you retweet someone, you get that protection as well.

      All of these things, and all of these other sites, and all these other services, and everything that everyone is building — I mean lots of people listening to this are building different internet services — all of those services are protected by 230. And this matters waaaaay beyond just the big three or four companies out there.

      Sonal: I am so glad you brought that up, Mike, because the most and really only alarming line in the executive order to me was this quote: “For purposes of this order, the term ‘online platform’ means any website or application that allows users to create and share content or engage in social networking or any general search engine.” And that is quite literally every site.

      Mike: That is every site.

      Sonal: Every site of every size. And it makes me think of the other law — it’s not Masnick’s Law of Impossibilities — it is the Law of Unintended Consequences.

      And this seems true for every regulation — and I think of GDPR and all these other regulations — that all they really did, in fact, was help bigger companies, the very group they were trying not to. All the smaller players who don’t have huge compliance arms, legal officers, and the people they can hire to moderate, process queries and takedown requests get punished, which then further entrenches [them]. So it’s a vicious loop, essentially.

      Mike: And that should be very scary. Because part of the executive order itself starts out by claiming that the reason they have to do this executive order is because there are limited number of social media sites out there.

      And yet the definition that they have in the setup of what they’re trying to do would effectively limit that, even further, by making it impossible for new competition to show up, and for smaller sites to exist. And the more you put in place these kinds of rules and regulations, the more difficult you make it for there to be any new startups in this space, any new websites — because it becomes a costly mess for any smaller website to comply.

      Sonal: Right. And while I completely agree with you that people alone or technology alone is not the answer, one thing I do want to point out about the “way forward” part of it is that this conflates the ownership of WHO decides versus the size of the company that decides.

      So, for instance, instead of having like a single CEO decide, “This is my vision for this big company,” crypto is an often cited case — my partner Chris Dixon, has written an op-ed in WIRED about this a couple years ago — as a way forward for thinking about the governance of some of these sites and thinking of a crypto-decentralized native way, so that it’s “a community owned and operated service” (which is his way of thinking about it). You and I have talked about crypto many many times over the course of our friendship and years (and I think at the inaugural Copia Policy Institute, I think you had a whole section on crypto, if I remember); and I’m curious for your thoughts on that as well.

      Mike: Yeah so last year, I wrote a paper for the Knight First Amendment [Institute] at Columbia University, which is called “Protocols, not Platforms.”

      Sonal: Ahh, I remember this.

      Mike: Oh yeah, the horcruxes.

      Sonal: I teased you about it where I was like, “Mike, hallows, not horcruxes Mike!” And I myself do not love when people use Harry Potter analogies, but my god that was so perfect for that. I’m sorry. It’s very much ”Hallows, Not Horcruxes” which is great — “Protocols, Not Platforms.”

      Mike: Yeah, you know, that paper discusses what the content-moderation world looks like in a distributed, decentralized system — potentially based on crypto. The paper touches on not just crypto, but just more decentralized, interoperable protocol-based systems.

      And that changes a number of the content moderation questions. It doesn’t make them go away — and I do think that is one mistake that some people make, which is they think if we just set it up on a crypto-based distributed system, then we just wipe our hands of it; and it’s everybody’s individual decision, however it’s implemented, let that happen.

      Sonal: It also doesn’t leave room for the variety of governance approaches that are inevitable in that as well. Because for the record, just as you’re arguing for a variety of experiments — whether it’s a privately owned, public owned company, centralized, decentralized, whichever — even in the crypto world, there’s a variety of governance approaches that can be applied, which is great. And there’s been a lot of experiments already playing out on that front when it comes to protocols.

      Mike: And I think that’s good! It is that experimentation that we need.

      And that experimentation is not designed just to like find the best result, but to recognize that there are different best results for different communities, and different purposes, and different services. There are certain cases where you want a Wikipedia approach; and there are certain cases where you want a Reddit approach; and there are certain cases where you want a Twitter approach; and whatever other approaches there are as well.

      You can have all these different things, and some of them work [only] in some cases. The only way we’re allowed to figure that out is if we have the freedom to make those choices and see what happens.

      Sonal: That’s a wonderful note to end on.

      So, in this show, we ask our guests (our experts) to bottom-line it for me. And while this has been longer than 16 minutes — it’s a special long episode — bottom-line it for me, Mike. What’s the big takeaway?

      Mike: So, the rules of how the internet works are under attack. This executive order by itself is not going to effectively change anything directly: It’s going to cause a lot of heat and light, but very little actual fire.

      What we are seeing — and this goes beyond just this executive order — is that, people are really trying to change the way moderation works online. And we’ve already seen some laws — both in the U.S., and certainly outside the U.S. there have been a bunch of laws that are direct to that content moderation — and that is going to continue. I worry very strongly about what that does, whether that locks everyone into a specific type of content moderation, and what that means over the long term for freedom of speech on the internet.

      Sonal: Thank you so much for joining this segment, Mike.

      Mike: Thank you, for having me.

      • Mike Masnick

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      SaaS Go-to-Upmarket

      David Ulevitch, Kristina Shen, and Das Rush

      Many SaaS startups often find their initial product market fit with small and medium businesses (SMBs) and then move up market. Because it’s easier to move up rather than down market, this go-to-market motion has often given SaaS startups the advantage against incumbents, who already have the added price and complexity of traditional enterprise selling.

      But moving up market comes with its own challenges.

      In this episode, David Ulevitch and our newest enterprise general partner Kristina Shen look at the SaaS go-to-upmarket motion. They cover why it’s easier to move up market than down, when to make the move, and how to price for it, including why so many founders underprice, deciding on free versus paid trials, and navigating the transition to larger accounts.

      Show Notes

      • Fundamentals of the up-market approach [0:51]
      • Questions around timing [3:00], how the up-market strategy developed [5:09], and what the future may hold due to remote work [6:29]
      • Discussion of pricing approaches [7:49] and mistakes firms make around pricing [14:57]
      • Important metrics to consider [16:39] and a discussion of free vs. paid trial pricing [17:50]

      Transcript

      Das: Hi, and welcome to the a16z podcast. I’m Das, and in this episode, I talk SaaS go-to-market with David Ulevitch and our newest enterprise general partner Kristina Shen. The first half of the podcast looks at how remote work impacts the SaaS go-to-market and what the smartest founders are doing to survive the current crisis. The second half covers pricing approaches and strategy, including how to think about free versus paid trials and navigating the transition to larger accounts. But we start with why it’s easier to move upmarket than down… and the advantage that gives a SaaS startup against incumbents.

      Up-market fundamentals

      David: If you have a cohort of customers that are paying you $10,000 a year for your product, you’re going to find a customer that self-selects and is willing to pay $100,000 a year. Once you get one of those, your organization will figure out how you sell to, how you satisfy and support, customers at that price point and that size.

      But it’s really hard for a company that sells up market to move down market, because they’ve already baked in all that expensive, heavy lifting sales motion. And so as you go down market with a lower price point, usually, you can’t actually support it.

      Das: Does that mean that it’s easier for a company to do this go-to-market if they’re a new startup as opposed to if they’re a pre-existing SaaS?

      Kristina: It’s culturally very, very hard to give a product away for free that you’re already charging for. It feels like you’re eating away at your own potential revenue when you do it. So most people who try it end up pulling back very quickly.

      David: This is actually one of the key reasons why the bottoms up SaaS motion is just so competitive, and compelling, and so destructive against the traditional sales-driven Saas motion. If you have that great product and people are choosing to use it, it’s very hard for somebody with a sales-driven motion, and all the cost that’s loaded into that, to be able to compete against it.

      If you have that great product and people are choosing to use it, it’s very hard for somebody with a sales-driven motion, and all the cost that’s loaded into that, to be able to compete against it. —David Ulevitch

      There are so many markets where initially, we would look at companies and say, “Oh, well, this couldn’t possibly be bottoms up. It has to be sold to the CIO. It has to be sold to the CSO or the CFO.” But in almost every case we’ve been wrong, and there has been a bottoms up motion.

      The canonical example is Slack. It’s crazy that Slack is a bottoms up company, because you’re talking about corporate messaging, and how could you ever have a messaging solution that only a few people might be using, that only a team might be using? But now it’s just, “Oh, yeah, some people started using it, and then more people started using it, and then everyone had Slack.”

      Kristina: I think another classic example is Dropbox versus Box. Both started as bottoms up businesses, try before you buy. But Box quickly found, “Hey, I’d rather sell to IT.” And Dropbox said, “Hey, we’ve got a great freemium motion going.” And they catalyzed their business around referrals and giving away free storage and shared storage in a way that really helped drive their bottoms up business.

      Questions of timing

      Das: It’s a big leap to go from selling to smaller customers to larger customers. How have you seen SaaS companies know or get the timing right on that? Especially since it does seem like that’s really related to scaling your sales force?

      Kristina: Don’t try to go from a 100-person company to a 20,000-person company. Start targeting early adopters, maybe they’re late stage pre-IPO companies, then newly IPO’d companies. Starting in tech tends to be a little bit easier because they tend to be early adopters.

      Going vertical by vertical can be a great strategy as well. Targeting one customer who might be branded in that space, can help brand yourself in that category. And then all their competitors will also want your product if you do a good job. A lot of times people will dedicate a sales rep to each vertical, so that they become really, really knowledgeable in that space, and also build their own brand and reputation and know who are the right customers to target.

      Das: So right now, you’ve got a lot more people working remote. Does this move to remote work mean that on-premise software is dying? And is it accelerating the move to software as a service?

      The old school way of doing SaaS go-to-market is bundle everything together, make the pricing super complex. New modern SaaS pricing is keep it simple, keep it tied to value, and make sure you’re solving one thing really well. —Kristina Shen

      Kristina: This remote work and working from home is only going to catalyze more of the conversion from on-premise over to cloud and SaaS. In general, software spend declines 20% during an economic downturn. This happened in ’08, this happened in ’01. But when we look at the last downturn in ’08, SaaS spend actually, for public companies, increased, on average, 10%, which means there’s a 30% spread, which really shows us that there was a huge catalyst from people moving on-premise to SaaS.

      David: And as people work remote, the ability to use SaaS tools is much easier than having to VPN back into your corporate network. We’ve been seeing that, inside sales teams have been doing larger and larger deals, essentially moving up market on the inside, without having to engage with field sales teams. In fact, a lot of the new SaaS companies today rather than building out a field team, they have a hybrid team, where people are working and closing deals on the inside and if they had to go out and meet with a customer, they would do that. But by and large, most of it was happening over the phone, over email, and over videoconferencing.

      And all the deals now, by definition, are gonna be done remote because people can’t go visit their customers in person.

      Das: So with bottoms up, did user behavior and buyer behavior change, so the go-to-market evolved? Or did the go-to-market evolve and then you saw user and buyer behavior change? I’m curious with this move to remote work. Is that going to trigger more changes or has the go-to-market enabled that change in user behavior, even though we see that change coming because of a lot of forces outside of the market?

      Kristina: I definitely think they are interrelated. But I do think it was a user change that catalyzed everything. We decided that we preferred better software, and we tried a couple products. We were able to purchase off our credit card. And then IT and procurement eventually said, “Wow, everyone’s buying these already, I might as well get a company license and a company deal so I’m not paying as much.”

      While obviously software vendors had to offer the products that could be self-served, users started to realize they had the power, they wanted to use better software, they paid with their credit cards. And now software vendors are forced to change their go-to-market to actually suit that use case.

      Das: If that’s the case that when user behavior has changed, it’s tended to be the catalyzing force of bigger changes in the go-to-market, what are some of the changes you foresee for SaaS because the world has changed to this new reality of remote work and more distributed teams?

      The influence of remote work

      David: We’re in a very uncertain economic environment right now. And a couple of things will become very clear over the next 3 to 9 to 15 months — you’re going to find out which SaaS products are absolutely essential to helping a business operate and run, and which ones were just nice to have and may not get renewed.

      I think on the customer, buying side, you’re very likely to see people push back on big annual commitments and prefer to go month-to-month where they can. Or you’ll see more incentives from SaaS startups to offer discounts for annual contracts. You’re going to see people that might sign an annual contract, but they may not want to pay upfront. They may prefer to meter the cash out ratably over the term of the contract.

      And as companies had empowered and allowed budget authority to be pushed down in organizations, you’re gonna see that budget authority get pulled back, more scrutiny on spending, and likely a lot of SaaS products not get renewed that turned out to not be essential.

      Kristina: I think the smartest founders are making sure they have the runway to continue to exist. And they’re doing that in a couple of ways. They’re preserving cash, and they are making sure that their existing customers are super, super happy, because retaining your customers is so important in this environment. And they’re making sure that they have efficient or profitable customer acquisition. Don’t spend valuable dollars acquiring customers. But acquire customers efficiently that will add to a great existing customer base.

      The smartest founders are preserving cash, and making sure that their existing customers are happy, because retaining your customers is so important in this environment. —Kristina Shen

      Pricing approaches

      Das: To go into pricing and packaging for SaaS for a moment, what are some of the different pricing approaches that you see SaaS companies taking?

      Kristina: The old school way of doing SaaS go-to-market is bundle everything together, make the pricing super complex, so you don’t actually understand what you’re paying for. You’re forced to purchase it because you need one component of the product.

      New modern SaaS pricing is keep it simple, keep it tied to value, and make sure you’re solving one thing really, really well.

      David: You want to make it easy for your customers to give you money. And if your customers don’t understand your pricing, that’s a huge red flag. Sometimes founders will try to over engineer their pricing model.

      Kristina: We talk a lot about everything has to be 10X better than the alternatives. But it’s much easier to be 10X better when you solve one thing very, very well, and then have simple pricing around it.

      I think the most common that most people know about is PEPM or per employee per month, where you’re charging basically for every single seat.

      Another really common model is the freemium model. So, think about a Dropbox, or an Asana, or a Skype, where it’s trigger based. You try the product for free, but when you hit a certain amount of storage, or a certain amount of users, then it converts over to paid.

      And then you also have a time trial, where you get the full experience of the product for some limited time period. And then you’re asked if you want to continue using the product to pay.

      And then there’s pay as go, and particularly, pay as you go as a usage model. So, Slack will say, “Hey, if your users aren’t actually using the product this month, we won’t actually charge you for it.”

      David: The example that Kristina made about Slack and users, everybody understands what a user is, and if they’re using the product, they pay for it, and if they’re not using it, they don’t pay for it. That’s a very friendly way to make it easy for your customers to give you money. If Slack came up with a pricing model that was like based on number of messages, or number of API integration calls, the customer would have no idea what that means.

      Kristina: There’s also the consumption model. So Twilio only charges you for every SMS text or phone call that you make on the platform any given month. And so they make money or lose money as your usage goes. The pricing is very aligned to your productivity.

      David: Generally, those are for products where the usage only goes in one direction. If you think of a company like Databricks, where they’re charging for storage, or Amazon’s S3 service, it is very aligned with the customer, but it also strategically aligns with the business because they know the switching cost is very high, the churn is very low. And generally, in those businesses, you’re only going to store more data, so they can charge based on usage or volume of data.

      Kristina: Recently, there’s been a huge trend of payment as a revenue. It’s particularly common in vertical markets where SaaS companies are adding payments as a revenue in addition to their employee or subscription revenue. If you look at Shopify, for example, more than 50% of their revenue is actually payment revenue. They’re making money every single time you purchase something off one of their shopping cart websites.

      Das: When you’re working with a founder or a SaaS startup, how have you seen them find the right pricing model for their product, for their market?

      Kristina: Step one is just talk to a lot of customers. Try to figure out what is the market pricing for possible alternatives or competitors, understand their pain points and their willingness to pay. And just throw a price out there, because you have to have a starting point in order to actually test and iterate. Particularly in the SMB, or the bottoms up business, you can test and iterate pretty quickly because you have so many data points.

      David: I always tell founders, step one is to just go out there and talk to customers. Step two is just double your prices. I don’t think there’s ever been a great company with a great product that’s fallen apart because their pricing was wrong. But a lot of SaaS startup founders really under price, and you don’t want to find out two or three years later that you were 200% underpriced.

      A very common thing that SaaS companies do, they’ll have the basic package that either is free or low cost, that you can just sign up online for. They’ll have a middle package where they share some pricing, and then they’ll have the enterprise package where you have to contact sales to find out more. And that way they don’t actually have to show the pricing for that third package. And that gives the salespeople the flexibility to adjust pricing on a per deal basis.

      Das: When you’re working with companies, why are they underpricing their products?

      David: I think it’s psychological. People need to price on value, and they don’t know how much value they’re delivering relative to “Oh, it only cost me $100 a month to provide this service, so I just need to charge $200.” But if it turns out you’re saving your customer $50,000 a year, then you’re wildly underpriced.

      You have to remember that SaaS is essentially a proxy for outsourced IT. You’re spending money on a SaaS service to not pay to develop something internally, or to have to pay IT to support something that’s more complex on-prem. Software is much cheaper than people, and so generally, the price point can be much higher.

      People need to price on value, and you have to remember that SaaS is essentially a proxy for outsourced IT. You’re spending money on a SaaS service to not pay to develop or support something internally. —David Ulevitch

      Kristina: And the other thing is your value increases over time. You’re delivering more features, more products, you understand the customer better. It’s the beauty of the SaaS model and cloud model that you can iterate and push code immediately, and the customer immediately sees value. A lot of times people have the same price point from the first customer sold to three years later and the 200th customer. Quite frankly, you’ve delivered so much value along the way that your price point should have gone up.

      The other thing I’ll say is a lot of people discount per seat pricing a lot as they move up market. We tend to tell people that the best validation of your product having great product market fit is your ability to hold your price point. So while there is some natural discounting on a per seat basis because people do deserve some volume discounting, I would say try to resist that as much as possible.

      Das: Especially for a technical founder, it’s so tempting to get in there and fiddle with these knobs. How do you know when it is time to experiment with your pricing and packaging?

      David: If you’re looking at your business and you see that you are doing more deals, and they’re closing faster, you should raise your pricing. And you pay attention to how long it takes to close deals and whether the number of deals is staying consistent as you do that. And, at some point, you’re going to find out when you’re losing deals on price.

      I think a moment where companies have to plan ahead to avoid having to course correct is after they roll out massive pricing and packaging changes, which are pretty natural as companies move up market. But how they navigate that transition to larger accounts, and how they either bring along or move away from those smaller, earlier customers who got them to where they are, tends to be really important because they can get a lot of noise on Twitter, they can get a lot of blowback from their customers.

      So Zendesk is a company where they rolled out a major packaging change. And when they rolled it out, they hadn’t planned on grandfathering in their early customers. They got a lot of pushback, and very quickly, they put out a blog post and said, “We hear what you’re saying, we appreciate you building the business that we’ve become today. We do need to have a package for the future. But all the people that have been customers so far will be grandfathered in for at least a period of time into the old model.”

      Kristina: If you iterate pricing constantly, you don’t really have this problem because your customers will be used to pricing changes. You normally pair them with new features, and it all kind of works out. But if you have to go through a big grandfather change, I tend to lean towards treating your early customers really, really well. They adopted when you weren’t a big company yet. They probably co-built the product with you in many ways. And so, it’s great to get more dollars out of your customer base, but treat your early customers well.

       A lot of people discount per seat pricing a lot as they move up market. The best validation of your product having great product market fit is your ability to hold your price point. —Kristina Shen

      Common pricing mistakes

      Das: Are there any other failure modes that you see startups really falling into around pricing and packaging or any common mistakes that they make?

      David: I think a lot of founders don’t always map out the cost or model of their pricing and their product relative to their cost of actually doing sales and marketing and customer acquisition.

      Kristina: Inside sales is so popular in Silicon Valley. When you’re selling more to an SMB or mid-market type customer, the expectation is that you’re educating and helping the prospective customer over the phone. And so, you’re not expected to be as high touch.

      But 5K is almost the minimum price point you need to sell to the SMB with an inside sales team in order to pay for the outbound costs and all the conversions, because there is typically a team that sits around the quota carrying rep. And so, price matching — how much your price point is compared to what your go-to-market motion is — matters a lot.

      Other big failure modes that I see, people guess the ramp time of a sales rep wrong. And ramp time really ties to the segment of customer you’re selling into. It tends be that if you’re selling into the enterprise, the ramp time for sales reps, because sales cycles are so long, tend to be much longer as well. They could be six months plus, could be a year.

      While if you’re selling more into SMB or mid-market, the ramp time to get a rep up and running can be much shorter, three to six months. Because the sales cycles are shorter, they just iterate much faster, and they ramp up much more quickly.

      David: The other thing that people have to understand is that sales velocity is a really important component to figuring out how many reps you should be hiring, whether they should be inside reps or field reps. If it takes you 90 days to close a deal, that can’t be a $5,000 a year deal, that has to be a $50,000 or even $150,000 a year deal.

      Das: Kristina, I know you’ve done a lot of work with metrics. So how do those play in?

      Kristina: Probably the one way to sum it all together is how many months does it take to pay back customer acquisition cost.

      Very commonly within the SaaS world, we talk about a 12-month CAC payback. We typically want to see for every dollar you spend on sales and marketing, you get a dollar back within a year. That means you can tweak the inputs any way you want. Let’s say that doing paid acquisition is really effective for you. Then, you can spend proportionally more on paid acquisition and less on sales reps. Vice versa, if you have a great inbound engine, you actually can hire a lot more sales reps and spend more on sales headcount.

      With all formulas, it’s a guide rail, so if you have customers that retain really, really well, let’s say you’re selling to the enterprise, and you’ve got a 90% or 95% annual retention rate, then your CAC payback could be between 12 and 24 months. But let’s say you’re selling to the SMB and churn is 2% or 3% monthly, which ends up being like 80% to 90% annual retention. Then, because your customer is less sticky, I would recommend looking at a CAC payback of 6 to 12 months.

      Free vs. paid trial pricing

      Das: How should you think about doing a free trial versus a paid trial?

      David: On the one hand, the bottoms up motion where people can try essentially a full version of a product before they buy it is extremely powerful. On the other hand, I’ve started to try to think about how I advise companies, when they are thinking about a free trial for something that might cost $100,000 or $200,000 a year? Do we do a paid pilot that has some sort of contractual obligation that if we meet then turns into a commercial engagement?

      Kristina: I do think the beauty of the bottoms up business is that you can get people to try the entire experience of the product for free, and they fall in love with it, and a certain percentage will convert. And that works really, really well for products that can self-serve.

      When you start moving up market to more complex products, the challenge with trials is it takes work to actually implement the product, whether it be integrations, IT has to give access, etc. You lose that self-serve ability, which is so amazing in the trial.

      And so, I tend to be more in the camp of paid trials, if it costs you money to actually deploy the trial. And when you’re selling to bigger customers, they associate value when they have to pay. Once a customer has to pay you, then they feel a need to make the project successful and thus they will onboard, schedule things, give you data and access.

      David: If you can get to a point where you get the customer to do that paid pilot, such that the only difference between a pilot and an actual customer is just the signing of a contract, that’s very powerful.

      Now, that does force you to have a really good pre-sales motion to make sure that you can deliver on the promise you’ve made your customers. When companies don’t have a great product, and they paper over it with professional services and sales engineering and post-sales support, that paid pilot thing doesn’t work because the experience isn’t good enough. So, it really is incumbent on the SaaS company that does a paid pilot to make sure that they are able to deliver on that experience.

      Kristina: And one emerging trend recently is people signing an annual contract with a one or three month out, as a replacement to the paid pilot. Because it’s the best of both worlds, the SaaS company that’s selling the product gets a higher level of commitment. And the customer gets the optionality of opting out in the same way as a trial without any clawback.

      It really comes down to where procurement falls. Sometimes procurement is at the beginning of that decision, which makes it more like an annual contract. Sometimes procurement is at the one or three month opt-out period, which means the customer already has a great experience, loves the product, and it is an easier way to convert procurements to actually sign on…

      David: And that is a really good segue into renewals. I always tell founders, you might have this subscription business, but it’s not a recurring revenue business until the second year when the revenue actually recurs.

      I think you really have the first three months to get a customer up and running and happy. And if they’re not, you then have about three months to fix it. And if all that works out, then the remaining six months of the contract can be focused on upsell and expansion.

      It’s not a recurring revenue business until the revenue actually recurs. You have 3 months to get a customer happy. And if they’re not, you have 3 month to fix it. Then you can have 6 months focused on upsell & expansion. —David Ulevitch

      Das: Awesome. Thank you, Kristina. Thank you, David.

      Kristina: Thanks so much for having us. This was fun.

      David: Yeah, a lot of fun, great topics, and our favorite thing to talk about.

      • David Ulevitch is a general partner at a16z where he invests in enterprise and SaaS companies. Prior to joining the firm, he was the founder and CEO of OpenDNS (acquired by Cisco).

      • Kristina Shen is a general partner at a16z where she invests in enterprise and SaaS companies. Prior to joining the firm, she worked at Bessemer Venture Partners, JMI Equity, Goldman Sachs, and Credit Suisse.

      • Das Rush

      The Environment, Capitalism, Technology

      Andrew McAfee, Marc Andreessen, and Sonal Chokshi

      It used to be that the only way for humanity to grow — and progress — was through destroying the environment. Sure, the Industrial Revolution brought about the growth of our economies, our population, our prosperity; but it also led to our extracting more resources from the planet, more pollution, and some nightmarish human conditions as well. But is this interplay between the two — of human growth vs. environment, of protection vs. destruction — really a zero-sum game? Even if it were true in history, is it true today? How about for developing economies around the world today — do they have to go through an extractive phase first before entering a protective one… or can they skip that phase altogether through better technology (the way they leapt to mobile)?

      And if capitalism is not responsible for environmental degradation, than who or what is? Where does technology come in, and where doesn’t it — if you believe we already have the answers to saving the environment? Marc Andreessen and Sonal Chokshi interview MIT economist Andrew McAfee about all this and more, given his new book, More from Less: The Surprising Story of How We Learned to Prosper Using Fewer Resources — and What Happens Next.

      So what does happen next? From nuclear power to dematerialization to Tesla and the next cleantech revolution (or not), this episode of the a16z Podcast brings a different perspective to an important discussion around taking care of our planet… and also ensuring human progress through the spread of human capital and technology.

       

      image: Kevin Gill / Flickr

      Show Notes

      • Overview of progress from the Industrial Revolution to today [1:22]
      • Definition of capitalism [5:47] and why it promotes efficient use of resources [11:22]
      • The benefits of clean nuclear energy [17:19]
      • Advantages of cap and trade to curb carbon emissions [19:00]
      • Debate over the future of resource use [27:21] and Fuller’s predictions about dematerialization [30:35]
      • New technologies needed to continue dematerialization trends [36:40]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. And I’m excited to do another one of our co-hosted episodes with Marc Andreessen, who joins me in interviewing MIT economist Andrew McAfee — who we’ve actually had on the podcast a couple years ago on a great episode with his coauthor, Erik Brynjolfsson, on their book, “Machine, Platform, Crowd.” But Andy’s new book takes a very different turn from that previous series of books — and focus, on the beam of bits — to focusing on atoms, the physical world. Basically the environment. It’s called “More from Less: The Surprising Story of How We Learned to Prosper Using Fewer Resources and What Happens Next.” And I think it’s a really important book, contributing to the important dialogue we’re having right now on taking care of our planet, and also of taking care of human progress — especially because these two don’t have to be a zero-sum game of the two in conflict with each other.

      So, what does it take to go from that narrative of extraction and destruction to one of protection and progress? So, in this episode, we cover everything from what capitalism’s role is in all this — including what it is and isn’t — to the global environment, including China and India. And throughout, we discuss the technology, from energy use and types of energy, to dematerialization — and, surprisingly, the idea of that well before software was even invented. Stay tuned for that bit. But we quickly begin with the technology and effects of the Industrial Revolution.

      From the Industrial Revolution to today

      Andrew: The industrial era kicked off with the Industrial Revolution, and the James Watt steam engine, and all those other technologies — was this period of amazing human growth. The growth of our economies, growth of our prosperity, growth of our population. And that was great in a sense, but it really did feel like there was a trade-off between improving the human condition and improving the state of the world. And in the industrial era, if you looked at the evidence, you could make a pretty strong case that we were increasing our growth at the expense of the planet that we all lived on. We took more resources from the earth every year. We chopped down more trees. We cleared more cropland. We took more fossil fuels out of the earth. We polluted more. We either domesticated animals or drove them to the brink of extinction. And the reason I decided to write “More from Less” is, I don’t think that’s true anymore.

      The evidence supports the idea that in the richest countries — and I’ve got the best data for America — that that trade-off between the human condition and the state of the world is actually in the rearview mirror. Because in almost all the ways that we could care about — improving the human condition — we’re taking fewer resources from our planet, we’re polluting it less. Some of the animals that we pushed to the brink are coming back. I didn’t hear that story being told. And so hence the book.

      Sonal: So one thought that struck me in looking at the example of the Industrial Revolution, which everyone points to as the greatest story of progress — you point out that it ended slavery but increased child labor?

      Andrew: There were some pretty nightmarish situations early in the industrial period. There really were factories full of kids under the age of 10, working 14-hour days.

      Sonal: Yeah.

      Andrew: And some of these kids weren’t even sent there by their parents. They were orphans. And this was what we decided to do. I consider that a moral mistake, and different than what kids were doing on farms before. But in most rich countries, slavery ended early in the industrial era. Child labor ended before the 20th century. But we didn’t start dealing with pollution and species that we pushed to the brink of extinction until much, much later than that. So we kind of looked after ourselves first and then the rest of the planet afterward.

      Sonal: So, Andy, I want to probe the conflation of capitalism and extraction of resources when it doesn’t actually have to necessarily be that way. But one stat in particular that struck me on that front is that research emerged showing that the U.S. GDP was closely intertwined with energy consumption. You talk about this in your book. Clearly, there’s something about more energy consumption tied with the success of an economy.

      Andrew: If you draw a graph of the U.S. economy — the real GDP of the U.S. — from 1800 to 1970, and then you add one more line to that graph, which is total energy consumption per year from 1800 to 1970 — those lines are really hard to tell apart. They sit right on top of each other. And there’s this whole stream of research that turned into an assumption — that if you told me what your energy use per capita was, I would tell you what your GDP per capita was or the state of advancement of your society. And we almost use those two things as proxies for each other. One of the super weird things is that that relationship has completely broken down in the United States — where, again, I know the evidence really well. Total U.S. energy consumption has been basically flat since at least the end of the Great Recession, and maybe even before that started.

      Now, in the old-fashioned way of looking at things, you say, “Oh my God, there was this massive recession.” Absolutely not, it grew like crazy. But we’ve divorced energy use from growing the economy. And one of the broad points I make in the book is — that story is very broad. We’ve divorced most other kinds of using up atoms — using materials — from our prosperity growth. And that relationship is not unique to America. It exists elsewhere, and it will spread as we spread capitalism and technology. 

      One of the things I have fun with in the book was trying to defuse tension, because there are a lot of audiences where if you say capitalism, they start throwing rotten tomatoes at you. They just can’t hear the word. It’s so triggering. So one thing I tried to do is say, like, “What do I mean by capitalism?” And I don’t mean cronyism. I don’t mean corporatism. I don’t mean regulatory capture, or financialization. These are all real things. These are all perversions of actual capitalism.

      Sonal: Yeah, I hate that capitalism gets a bad rap. And while we may argue for a better form of capitalism, can you just break it down and sort of tease apart the myth from the facts when it comes to, like, what is capitalism? I think sometimes people are using different labels for different things quite honestly.

      Andrew: Yeah. And let’s be clear on what we’re cheerleading about. Capitalism is the best way the earth has ever come up with to get goods and services into the hands of people. Now, that’s a really important thing for a society to do, if you don’t want your people to starve or die of exposure. And when I talk about that, I mean a few pretty specific things. First is that private companies are responsible primarily for producing those goods and services. It’s not the government. It’s not individual craftsmen or artisans or anything. Number two, they use prices that are not centrally set or controlled. And prices convey a huge amount of information about abundance and scarcity, and where you should allocate your attention.

      So, we really need prices to be floating around in an economy. We need your property rights and your contracts to be respected by a working court system that believes in protecting those things. So that, if you’re an upstart, if you have a good idea — either the government or some big powerful company, or some billionaire, can’t just come and take it from you without compensating you and without your agreement on that stuff. One of the most important phrases for capitalism is voluntary exchange. You can’t force me to sign a contract. You can’t make me buy a product or forbid me from buying a product. You can’t stop me from moving to another state. So you just have this — it’s free-flowing. But there are these hard and fast constraints and rules about it. If you get those things right, the goods and services will become abundant to people.

      One of the things I loved writing the book was that Adam Smith nailed all of these topics in 1776. And yet here we are almost 250 years later, and we’re arguing about things that he kind of put to rest a long time ago. He said you need actual competition, not cronyism, for the benefits of capitalism to accrue. Amen to that.

      Marc: He actually went further. He actually called business people the enemies of capitalism.

      Sonal: Why?

      Andrew: <laughter> He’s got that famous quote that, “Men of trade seldom meet together, even for merriment, except it winds up in a conspiracy against the public.”

      Marc: Yeah, he basically argued, it’s like what modern libertarians are actually arguing, which is basically to the extent that business people, like, start to get involved in political policy. They try to rig the political system in their favor. And then that trips the line between so-called capitalism and corporatism. Then politically, that’s sort of the distinction between being pro-business and being pro-market.

      Andrew: Or being pro-incumbent and pro-market.

      Marc: Right, right, exactly. And what you want is you want to be pro-market. This is what we run into in our business. You know, because we launched these new companies that don’t have any political power whatsoever. And they go into these industries that in some cases are heavily dominated by incumbents. And invariably what you find is an intertwining of the incumbents with the regulatory system, often under the color of consumer protection. But it actually turns out, what’s happened is the incumbents have rigged the system. They’ve rigged the politics for their own preservation, and the hypocrisy gets exposed in the form of, like — you just have a product that’s just obviously better. And then the captured regulatory state comes to try to shut it down to protect incumbents.

      Andrew: Well, my favorite one of that was — for a while, I think France or Paris had the requirement that a limo had to go back to its home station for 15 minutes before picking up another customer. Why on earth would that be?

      Marc: That’s right? Well, this is always the absurdity of would you really rather, like, stand out in the rain with your arm up, seriously? Right? And by the way, would you really rather have a system in which the driver is able to, like, eyeball you in the street and decide to not give you a ride?

      Sonal: I was about to say it because what people don’t talk about is the disproportionate impact on people who don’t look like they’re someone who you want to give a ride to. And now you can get a ride anywhere by anybody.

      Marc: Yeah, exactly. But then there’s this risk that, you know, you become the thing that you hate, you know — which is always a danger.

      Andrew: We also need to acknowledge there are problems that capitalism itself doesn’t solve. People getting left behind, inequality of some kinds of opportunity, the lack of a safety net, pollution, species loss. Absolutely, these are well understood — sometimes called market failures — and we need to be thoughtful about those things. But, again, Adam Smith, I don’t think he talked about species loss and extinctions, but he got these things right in 1776. And it kind of frustrates me that there’s still this big Marxist hangover going on, where people willfully or ignorantly misunderstand this economic system that we have.

      Marc: So in the 20th century, were capitalist systems or communist systems worse for the environment?

      Andrew: Oh my God, there’s no comparison. I think the single saddest and most tragic story that I learned when I was researching the book is about the Soviet whaling industry. The Soviets signed up for all the treaties to sharply limit the whale hunts. And then they ignored the treaties that they signed, which is bad enough — and they killed about 200,000 additional whales over the decades before they finally stopped. The crazy part of the story is why they killed 200,000 additional whales. And the answer is for no good reason at all. They didn’t eat the meat. They didn’t need the blubber, because they were already self-sufficient in oil. The only reason they did it was because they had Stalinist five-year plans for growth in the fisheries industry. And whales weigh a lot. And if you kill lots of whales, you grow your fisheries industry.

      There’s this heartbreaking story about the guy that was in charge of the fisheries industry. And he was such a pro at executing Stalinist five-year plans, he was named Hero of the Soviet Union. And one of the Soviet scientists went to him at some point and said, “We have to stop this. There will be no more whales for our children to see.” And his reply was, “Our descendants will not be the ones to fire me from my job.” <Damn.> So, you know, we can talk — the capitalist systems, we made pollution mistakes, yes, and we corrected them. What closed communist systems did was keep making mistakes under cover of darkness for no good reason.

      Capitalism’s effect on emissions

      Marc: I mean, this is very relevant to current events, right? One of the things that is very common, obviously, in the United States right now, is the theory that capitalism is responsible for environmental degradation — and unless we convert to a socialist system immediately, like, the environment is doomed. And, therefore, the very clear assumption and statement is that shifting to a socialist command and control system will lead to better environmental outcomes. That’s a very common theme right now. Like, how do you address that in present times?

      Andrew: Yeah. And it’s a tiny bit hidden, right? Because the people who make that argument — I hear them railing against capitalism and saying we need to take better care of the planet via some alternative. And then they get kind of vague about what that alternative is. But I think they’re all either dodging the fact, or lowballing that they want central planning. They want a command and control economy. And let’s call that what it is. It’s something between socialism and communism. And the thing that we need to keep in mind is that the capitalists — the free societies of the West — were the ones that dealt with their pollution problem earliest and best. And what I consider the great triumph of the environmentalist movement that, you know, kicked off around Earth Day, was that “we, the people” demanded that we stop having polluted air and dirty water and things like that. And we got it via things like the Clean Water Act, the Clean Air Act, the Marine Mammal Protection Act. These were landmark pieces of legislation.

      The single most important thing that happened after the legislation was passed was we got clever about how to reduce pollution levels. The story of cap and trade for reducing particulate emissions from power plants and reducing sulfur dioxide is such a fantastic story, because we put together this coalition of environmentalists and conservative economists. And we put in place a market system for trading pollution — which sounds weird and bad, except that it has cratered our levels of SO2 and other particulate pollution, and done it for about one-fifth of the originally estimated cost. It was just this extremely efficient thing to do. So the notion that capitalist systems have no way of dealing with increasing pollution is just dead flat wrong.

      Sonal: When I was reading the book, one thing that struck me was — do you think that with developing countries today, like India, China. I mean, one would argue they’re more developed than fully developing, however you define it —that they even have to go through an extractive phase first, in their first phase of figuring out how to use their resources? Like, I guess my question is why couldn’t they leapfrog this extractive phase, and just go right to a more practical phase when it comes to the acceleration of technology? Do you think that extractive phase has to happen?

      Andrew: It’s pretty clear to me that America and the UK, and I think most other super-rich countries, are past peak stuff. If you weighed our economy year after year, it would weigh less year after year. India and China and Bangladesh are not yet at peak stuff. But they will get to that point much earlier in their GDP per capita trajectory. Because, you know, Nigeria is not going to lay an extensive copper telephone network across the country. They’re not going to build as many coal plants per capita as we did, because that’s just economically inefficient to do. I’ll be surprised if the Chinese have as many private cars per capita as we did earlier in our history, because it’s really impractical to have that heavy, expensive asset sit idle 95% of the time.

      So I do think that this technologically very sophisticated economy is going to get countries through this resource transition much earlier than we went through it.

      Marc: So one of the things that’s so striking — carbon emissions, right, in the U.S. are falling. And are you telling me they’re starting to fall in certain parts of Europe as well?

      Andrew: Yeah, the EU, in general, has been on a shallow downward trend.

      Marc: Yeah, you know, there’s lots of advances being made in energy-efficient, you know, technologies of all kinds. And so one would imagine, like, this will continue. Let’s take the strong advocates for dramatic action at their word that we’re gonna run into real trouble globally. How do you not progress from there, to believing we have to take a very different approach from a foreign policy standpoint — in particular towards China and India — potentially up to and including coercive actions? Because if you look at the graph of global emissions growth, it’s very clearly two, like, gigantic examples.

      Andrew: So we’re going to invade them to make them reduce their carbon emissions? I don’t see how that plays out. Let me give you a couple of softer ways, because I think there are a couple important ones. One is, they gave the Nobel Prize to Bill Nordhaus last year for his work about how to deal with global warming and the notion of a carbon dividend. When Nordhaus proposed his carbon tax — and I like the phrase carbon dividend better, because it’s not a tax where the government keeps the money, you pass through the government directly to people and give them a carbon dividend, hopefully skewed a little bit toward lower-income people. As part of that, you also do what’s called a border adjustment, where you look at all the imports into the country, and if they come from high-carbon sources, you tax them — just like you would if they were made in this country with high-carbon sources.

      I think that’s a really strong incentive for our main trading partners. And China’s probably exhibit A here, to start literally cleaning up their act in this regard. The other thing is, you know, we have one source of power — we have one way to generate power that is scalable, clean, somewhat economical, and not intermittent. And it’s called nuclear. And there are a couple of countries like France and Sweden that have cheap electricity and the cleanest power in Europe. And we’re running away from — and the rest of the world. I find this completely perverse. Why not put together an international coalition — and along with that, an international patent bank — so that it’s cheaper to produce the new generation of nuclear reactor. I’m pretty sure that will get the cost down to the point where it becomes an economic no-brainer, even for low-income countries, to start transitioning into a clean energy environment.

      I would do both of those things way before I would try to coerce other countries into changing their energy profile, or doing it in a way that would slow down their growth or impoverish their people.

      Marc: So I’m glad you brought up nuclear. I was gonna ask you that. So many groups, just, like, flatly roll out nuclear as an option. So what’s going on there? And, like, what’s the way through that?

      Andrew: I honestly don’t know the answer. Why are they so stridently anti-nuclear? There’s probably a bundle of things going on. One is because of everything from Hiroshima and Nagasaki, to Godzilla, to Three Mile Island and Fukushima and Chernobyl. I mean, I just finished watching their “Chernobyl” miniseries on HBO. So I have this kind of visceral “ick” reaction to the idea of super widespread nuclear power. But I think our homework is always not to trust that initial “ick” and to go look at the evidence. And when you actually look at the evidence and look at the issues, I don’t know how you come away [as] anything except a nuclear advocate. And we worry about things like nuclear waste, and we should worry about nuclear waste.

      But we don’t then say, well, how much harm is caused by the pollution from other kinds of power generation? Worldwide, there are clearly hundreds of thousands of deaths a year from people breathing coal dust, and people breathing the emissions from coal plants. So the death toll — it’s not even close. And this is backed up by very good research published in “The Lancet” and elsewhere. There’s a nice article in “Our World in Data” about relative safety levels and death rates from different kinds of power. You walk away from that nuclear’s biggest cheerleader. So I don’t quite know why the reaction is so strident and visceral and negative. All I can say — it is not based on evidence. And I’m starting to see a coalition forming that pushes back against that to say, “We’re getting this deeply wrong on an important issue.”

      Success of cap and trade

      Sonal: Okay, so you were talking about cap and trade. What made that so successful compared to other attempts? Obviously, there’s a market-based mechanism, but give me more details.

      Andrew: Cap and trade — the basic idea is [to] make pollution expensive. Attach a cost to it. In other words, put it inside the market. Pollution doesn’t naturally have a price. And when that’s the case, no matter what the press release says, businesses have a strong incentive to go pollute, if it’s free. Okay, put a price on it. And then here’s the brilliance of cap and trade — allow companies to buy and sell that pollution, or more specifically, that right to pollute with each other. So if I’m super dirty, and I can’t clean up quickly, I’ve got to buy the right to pollute. But I’m willing to buy that right if it’s cheaper than the cost of me cleaning myself up.

      Sonal: Right.

      Andrew: Somebody will sell me that right and make some cash, because they’re already really clean, and they don’t need that right. So this was a line of economics research that got started with legendary Nobel Prize-winning economist Ronald Coase, and descendants of his ideas got put into practice early in the Reagan administration with the help of, like, the Environmental Defense Fund. So this beautiful alliance formed to say, “Hey, let’s try this market-based thing for dealing with pollution.” They overcame whatever reluctance was there from the incumbents, again, and they did it. And then the research is pretty clear that we can just look at what happened to particulate emission from these kinds of plants. America’s skies are just 90-plus percent cleaner than they were when that legislation was passed. And the cost of doing it is a fraction of the original estimate of that.

      So there’s a reason for these kinds of crazy fans of markets for getting things done — they work. And when you can put things like pollution in a market, and you do this with cap and trade and carbon dividends and things like that, these are the most efficient ways to deal with the problem.

      Marc: Don’t China and India have to sign up for the same thing?

      Andrew: One of the problems with carbon is that the harms from it are not local, and they’re not immediate. So maybe the fast-growing, high-carbon countries right now will choose to ignore it for a while longer. We have a couple mechanisms to get them to not do that. And like I said, if you do a border adjustment for the high-carbon products that we import, that’s a really strong incentive to do things better — if we can make it cheaper for them to be green. And, personally, I think nuclear and, you know — a patent banker, cheap technologies around nuclear — is the path to do that. We clearly have to help the currently low-income world get rich on a lower carbon trajectory than they’re on right now. That’s different than saying that they can’t use more energy year after year. I’m not gonna deny them that right to prosperity.

      Sonal: Exactly.

      Andrew: But we really want them to get cleaner quicker. I think we have tools to do that. And I don’t think that the Chinese and the Indians are indifferent to the longer-term health of the planet. I really don’t believe that.

      Sonal: I mean, they’re living with it in the physical way. Everyone there is facing it and experiencing it in a very real way. And we had this podcast a few years ago with Evan Osnos at “The New Yorker.” We were talking about China. One of my favorite things that he talked about is how, because of the growth of the middle class in China, that there is now a huge cohort of people demanding a better environment — precisely because of the market dynamics.

      Andrew: Not just that — getting a better environment. So I found this great research that I put in “More from Less.” A very good economist looked at what happened when China finally got serious about urban air pollution. And the reason they got serious about it was — people were leaving the cities even if they didn’t have government permission to do it. People were leaving because their kids were just clearly getting sick and going to have stunted lives. So China took action. And they brought down their country-wide particulate pollution by 30% in 4 years. And they did it with these draconian means, but they did it. And it took us in the United States 12 years to get that same 30% reduction.

      One of the points I make in the book is — democracies are probably more receptive to the will of their people. But there are interesting exceptions in both directions. And China was clearly receptive to the will of its people not to choke off their children with pollution.

      Sonal: Right, right. I read a ton of Chinese sci-fi, and it’s literally — the recurring theme is basically about the end of the world and, like, [the] environment. But, Andy…

      Andrew: Is that right? That’s cool.

      Sonal: Yeah, it’s a really big theme, and you have to read a lot of different Chinese science fiction authors to see this, but that’s basically my genre this year. One thing I want to ask you. I understand from the market dynamics point of view why cap and trade was such a successful idea and example, and it’s been proven out. But why couldn’t a government have simply mandated, like — we will just simply put a limit on this. Draconian measures like China did. Why would that not be as effective?

      Andrew: Sometimes we did. That’s how we actually brought down CFC emissions so drastically. We just mandated that they be reduced by X percent over time until they got down to close to zero. The reason that worked is that there’s a relatively small number of industries, a relatively small number of companies, and a relatively small number of products that used chlorofluorocarbons. And to be a little bit more cynical, the other reason that ban worked was — somebody eventually whispered to the incumbent companies, “The CFCs you’re making out right now, they’re off-patent. The new generation of coolants, and propellants, and whatnot — those can be under patent. Those can be a big revenue source for you.” And so they finally got industry on their side.

      Fiat can work. You know, for example, it is just flat [out] illegal to dump waste at sea in America. We just did that via fiat. We didn’t put a price on it. You cannot hunt animals in national parks. You cannot hunt deer or duck outside their seasons. So sometimes you want to do things by fiat. But I kind of think if you can put it in a market mechanism, and it’s appropriate to do that, I think you’ll get better solutions quicker. Maybe that’s not right. But I’ve got this deep faith in markets — once you put things in them and price them to deal with that price in a very fast way. If you change a business’ cost structure quickly — man, businesses will run from that increased cost like gazelles run when they smell a lion. It’s just amazing how quickly it’ll happen.

      Marc: I will tease — or torture Andy a little bit. And that is, you’re probably well aware — support for market-based systems like cap and trade have collapsed.

      Andrew: One of the points that I bring up in the book is that sometimes the crazy side of the argument wins. And I think the crazy is winning on nuclear these days. I think the crazy is winning on GMOs. I think the crazy is winning on vaccines in way too many communities. So, you know, as much as I love evidence and trying to think through things, we better be very good communicators about our solutions because the crazy can win.

      Sonal: Can we quickly talk on GMOs and the myths and misconceptions around GMOs? Why did you think it was important to talk about GMOs in your book?

      Andrew: The reason I thought GMOs were important to include in this book is, they are great ways to help us tread more lightly on the planet. The crop yields will go up. You can grow them in different places. As climate change happens, you’re going to need plants that are hardier — can survive heatwaves and droughts and things like that. The GMO toolkit is our best toolkit for accomplishing those things right now. And yet, it’s stridently opposed by governments and all kinds of groups around the world. And even the EU itself, in addition to the National Academies of Science, and just about every country that you can think of — has reviewed the evidence, and they’ve all come down and said there is no evidence that GMO crops are less safe for the environment, or for humanity, than conventional techniques.

      We can get past the point of saying, “Well, it remains to be seen.” No, we need to go do these things. And the reason I get exercised about this, is when I look at things like golden rice — which is this strain of rice that has beta carotene injected into it via GMO techniques, so that you provide vitamin —is it A? It’s a vitamin A deficiency, happens to babies who are weaned on rice gruel, and it leads to blindness. And that deficiency is responsible for about a million deaths a year around the world. Great, you’re anti-GMO? Honestly, that volume of deaths, that’s on you.

      Marc: So you discuss in the book, a very famous — at the time, I guess, in the ’70s and ’80s — a very famous debate between two, at the time, very accomplished people — Julian Simon and Paul Ehrlich. And it’s largely been forgotten, but it’s a highly relevant — and maybe even more relevant today than it was at the time. And maybe you could describe their debate and the famous bet.

      Debates over resource use

      Andrew: Yeah, my favorite bet of all time was the bet between these two guys. Julian Simon pushed back against the dominant narrative, around the time of Earth Day, which was that growth will come to a bad end — that we cannot keep this headlong, uncontrolled market-based growth for a bunch of reasons. Primary of which was — there’ll become too many of us, the earth will not be able to feed everybody, and we’re going to crash into a massive famine. And the prime exponent of that view was Paul Ehrlich, who still is at Stanford, and wrote a book called “The Population Bomb,” where he essentially said, “Look, nothing we can do will prevent hundreds of millions of people from starving in the years ahead. But if we do things like forced population control, and we take control of the means of production, we might be able to stave off the worst things that could happen.” And one of the things I learned was that Simon agreed with that, and wrote things about population control. Then he switched his view, in this wonderful instance of intellectual honesty and humility. And he said, “Wait a minute, we keep on not seeing famines happen, resource crises. We just don’t see these things. Instead, the evidence shows that most things are getting better.”

      And he got laughed out of a lot of rooms, and Ehrlich kept on putting out this gloom and doom, “stripping the planet” narrative. And, finally, Simon challenged him to a bet. And Simon said, “Pick any time period of at least a year, and pick any bundle of resources that you want. And at the end of the time period, if the resources are more expensive in real terms than they are now — which kind of means they’re more scarce than they are now, I’ll pay you the difference. If they’re cheaper, you pay me the difference.” I think this probably appeared like a sucker’s bet to Ehrlich. He picked five resources — tungsten, tin, chromium, copper, and I forget the fifth one. And he said, “All right, let’s put a 10-year period on the bet.”

      By 1990, the real prices of all five of those things had fallen. The price of the total portfolio had declined by more than half. And Ehrlich mailed Simon a check to acknowledge that he’d lost the bet — didn’t talk about it very much, didn’t attach any kind of note to that check. So I love that episode so much. And I’m trying to do round two of that. I’m using the Long Bets website, which is part of the Long Now Foundation, started by Stewart Brand and others. And I’m offering bets. I’m saying, for example, that no matter what — I’m saying that resources are going to become more affordable. I’m agreeing with Simon on that, but I don’t stop there. I say, in 10 years from now, I bet we’re gonna use less total energy — not per capita, but total energy America-wide in 10 years, after a decade of continued economic growth.

      That’s how confident I am in the one-two punch of capitalism and tech progress to take costs out of the system. And, you know, energy and resources cost money. That’s just my reasoning. If you think I’m wrong, step on up. With Long Bets, you both put the money up front. You designate a charity that will get it at the end, and we’ll see what happens.

      Marc: So there’s two historical figures in the book who are heroes of mine.

      Andrew: Julian Simon and…

      Marc: And Bucky Fuller. He came up with this idea, and I think you say it was 1927?

      Andrew: Yeah, the ’20s.

      Marc: Maybe just explain his idea. Because that was a remarkable insight at a time when there was probably no actual logical foundation to expect what he was saying.

      Andrew: So Fuller was this crazy polymath. And he popularized, for example, the geodesic dome — that’s kind of what he’s best known for today, I think — which is the structure that can bear a great deal of weight and very heavy loads while weighing very, very little. And Fuller thought that we would see more and more examples. And there were plenty of opportunities to do that kind of thing all around the economy. And I found, you know, this crazy book that he wrote in the ’20s. And he said, “Look, I did a bunch of calculations.” And he said, “I thought it might be possible to satisfy all of our wants and needs, essentially while using less stuff, while using fewer materials.” And he called the process ephemeralization, making things more ephemeral. That’s a real mouthful to say. So we use the phrase dematerialization more often now. But Fuller was the guy who said, “Gang, we can do this,” in the 1920s, which is crazy.

      Sonal: That’s so crazy. That’s pre-software.

      Marc: The economy in those days — it was what Joel Mokyr calls the “wheat and steel economy.” That was during the era where GDP first became an economic metric. And it was literally like tonnage.

      Andrew: We were weighing things, right?

      Marc: It was, like, how much you weigh your output, right? In tons.

      Andrew: Yeah. And then we started counting dollars instead, and that was a huge innovation. So the fact that Fuller came up with that that early, is just this weird intellectual shooting star.

      Marc: So if I recall correctly, and maybe I’ve made this up in my own head — but I think that one of the lines he used was, “Ephemeralization is the process of making more and more with less and less.” But then he added a line, he said, “Until eventually we are making everything with nothing.”

      Andrew: I think he did go that far. He also said, in 1927 — he said, “It’s the number one economic surprise of world man.”

      Marc: Right.

      Andrew: And so here we are, you know, 90 years later, and it’s still surprising to people.

      Sonal: So, one thing that just blew my mind, because I had not actually read that or known that. How could he come up with that in 1920? This is before software even existed. Like, what would give him — because I understand now, Marc, in 2009, when you wrote “Software is Eating the World” — like, I could see someone making that [claim] now. What gave him the chutzpah to say that in 1920? Like, that’s insane.

      Andrew: I have no earthly idea. And I don’t think we would have got to this resource turning point — I don’t think we would have achieved absolute dematerialization — without the digital world, without the computer.

      Sonal: Yeah.

      Andrew: I think software is giving us back the world because it’s letting us slim, swap, optimize, and evaporate our resource use. And I don’t know how we would have got there in a world where we’re still using slide rulers and file cabinets. Maybe we would have. But in my multiverse, we don’t get there in the universes that don’t have the digital revolution.

      Sonal: A lot of people when they talk about dematerialization, they talk about it very literally — like, you’re replacing an object, a hard object, with something — its software counterpart. But just make it clear, it’s actually even deeper than that, because when you do think about ride-sharing, and all these entire economies that are growing off the mobile phone — that is what enables the end of ownership. When you think about the fact that today kitchens can be delivering food to you, that is the thing that changes the shape of cities, etc. I think a lot of times when people talk about dematerialization, they take it very literally as, like — the one-on-one replacement of something physical with something digital, and it’s actually bigger than that. It’s like, a whole services economy and reshaping things.

      Andrew: Yeah. I talk about these four different vectors for dematerialization. You know, trimming out how much aluminum is in an aluminum can — that’s slimming it down. Swapping out one resource for another, that’s when rare-earths gets expensive. We walk away from them. Optimizing — using the load factor for airlines — has increased from the mid-50s percent to 80% now. You’re just making better use of these resource-intensive assets that you have — and then evaporate, replace it by nothing at all. The smartphone has made me not print out maps or print out film anymore. We have these different vectors for dematerialization to happen. And the point that I make in the book is — they’re happening in obvious ways, in subtle ways, in big ways and small ways, in the foreground and in the background in every industry. Simply because stuff costs money, competition makes you want to save money. And the digital toolkit offers you these great opportunities to do that. I think the story is just that simple. And if that’s true, it’s not about to end.

      Marc: So, if you take Fuller’s thought and your thought to their logical extremes, how close can you get — ultimately, someday — to making everything with nothing? Like, if we’re sitting here 50, 100, 200 years from now — like, what are the prospects for being able to take physical inputs out, you know, either 99.99% reduced or taken out entirely from many of the things we’ll be consuming?

      Andrew: That depends on how many of us there are, primarily, I think — but I think we can go a lot further down the dematerialization curve than we are right now. It’s not crazy at all to imagine that, you know — let’s say in 2100, that we’re primarily an urban species. We live in these densely populated cities that are, you know, a lot closer to Singapore than Delhi, for example. We’re growing a lot of our food in very vertical, energy-intensive environments. When we need to build a new building, we’re just recycling the steel and the metal that we used for the previous generation of buildings. We’re already doing that a lot right now. And, you know, grow our textiles in weird vats with Petri dishes of bacteria or something. That’s no longer crazy to think about. Will we be getting our protein from living animals, you know, or from scaled-up Petri dishes in 2100?

      Sonal: Lab-grown meat, yeah.

      Andrew: And who knows about staple crops, if we’ll need cropland for that? But I’m for damn sure that we’re gonna need a much, much smaller acreage of cropland for all of humanity in 2100 than we do right now. So I don’t know, I don’t have a good way to guesstimate where those lower floors are. They’re a lot lower than they are right now. And I really think that — let’s take 2100 as the year — we’ll be this species that occupies a very small physical footprint on the planet without depriving ourselves. And then we go into nature, kind of because it’s cool and because we want to, as opposed to because we need to strip it to satisfy our growth.

      Technology developments needed

      Marc: I have a question about R&D — the role of research and development in, kind of, delivering on the dream that you’re talking about. You know, because obviously, everything you’re talking about is sort of dependent on future development of advanced technology and creation of new knowledge. The last, like, 20 years — I would say, there’s been basically, like, two dramatic events in energy-related R&D in the U.S. And one is this incredibly positive outcome with respect to fracking and liquid natural gas. There’s been all kinds of positives to come out of that. And even in the energy industry, a lot of experts were shocked [by] how well that stuff has worked. The curves are amazing, because it’s like —  energy production in the U.S. [is] falling, falling, falling, falling, falling — and then, all of a sudden, it just, like, takes off like a rocket ship. Right? When like, nobody was expecting it.

      Andrew: To the surprise of everybody.

      Marc: Yeah. So that was the good news surprise. The bad news surprise was, you know, Silicon Valley embarked on a very big push to do so-called cleantech/greentech, particularly between 2010, 2012. There’s a huge push. And there were a lot of extremely smart and accomplished people here in the valley who thought that this was the new frontier for American technology, for venture capital and — you know, with obvious, you know, both huge potential positive benefits for the world but also, you know, a huge opportunity to build new businesses.

      And I think there were hundreds and hundreds, possibly even thousands of companies, and a very large amount of money and effort — and a lot of people put a lot of work into this, that the results were extremely disappointing on a number of fronts. I mean, there were, maybe, a few isolated cases of success. I mean, one might say we got Tesla and SpaceX out of that, right? In which case, you know, fair enough. But even beyond that, companies had a much harder time developing and/or commercializing those technologies, or just ended up in dire straits that people didn’t expect.

      So I’m very curious of your assessment of, like, what went wrong in the Silicon Valley cleantech/greentech adventure. And what should we learn from that, you know, both, like, as an industry and as, like, a world? If we’re going to try that kind of thing again, like — if we’re going to try to double down on R&D here, like, what lessons should we learn from that in terms of how to do it better?

      Andrew: I only know it from a great distance. Here’s a super naive way to look at it. If we think about solar, solar has become dominated by China, primarily, because it’s a flavor of manufacturing that they were already pretty good at. And it’s just a scale economies game. <Exactly.> And they’re quite good at scaling up huge factories and turning out, whether it’s a liquid crystal display or a photovoltaic panel. So I think that’s just very, very tough competition. The other thing that I do believe about solar and wind is that they have a place in the energy portfolio, absolutely. But Germany’s experience with trying to become much more reliant on renewables has not gone very well at all, for a couple of reasons — a deep one of which is — it’s dark sometimes, and it’s not windy a lot of times. We have this very serious problem of intermittency with those renewables.

      So, they have to be backstopped with something. And if you turn off your nuclear stations — if you decommission them, like Germany is doing — you get backstop in their case with some very dirty coal-powered plants. So they’ve kind of got the worst of both worlds. Their electricity prices are really high, and their carbon emissions, per unit of energy, are really high. You look next door at France, which is very nuclear, and you see neither of those two problems happening. So I think at the individual competition level, going up against China in a scale game is really, really tough. And I think there are some policy mistakes that can make that situation worse. Does that play at all with your experience?

      Marc: I think those are definitely big components. You may know the sort of appendix to that whole saga was, yeah — so there was a huge push for solar panels, including some very advanced — we actually have — here in the conference room, we actually have an old cylinder solar panel, one I keep around just because it’s such a great story. It’s the cylindrical solar panel, right? That would have a huge advantage that it could basically follow the sun. <Tracking.> You could track the sun. The only problem with it was it ended up being a 4x-worse value proposition — price-performance value proposition — than conventional solar panels, all in. That was one of the train wrecks out here that actually took down the whole U.S. government DOE program to fund cleantech.

      But the kicker on the whole solar thing is — okay, as you said, it became a mass manufacturing game. And so it kind of became, like, memory chips in the ’80s. It lent itself to the Chinese ecosystem, which is able to do mass manufacturing at scale.

      Andrew: Quickly and well.

      Marc: Right, right. Exactly. Right. Exactly. And so the Chinese have been able to undercut a lot of their American competitors. The kicker to that is the pro-environment administration then reacted to that by putting tariffs on imports of Chinese solar panels — therefore making it cost-ineffective for Americans to deploy solar panels that otherwise would have been much cheaper.

      Andrew: So tariffs are — with the possible exception of a border adjustment tariff, because we got to bring down carbon, right? Tariffs are just — Econ 101 bad idea.

      Marc: Well, it went beyond, though, just this specific mechanism. It was more an expression of values on the part of the United States government, which is — in theory we care about the environment. In practice, like, we’re more worried about, like, other things. And so we’ll trade off the environment.

      Andrew: Yeah. So, you know, the mantra is all — should be, “Let markets work to develop the goods and services and let free trade happen.” And that’s where prosperity will come from and innovation.

      Sonal: For me, I was in the thick of that, because we were at the heart of this whole cleantech movement. When I was at PARC, we had a huge investment in photovoltaics. It was my first big white paper. My question is, why can’t it just be just a timing thing, like everything else. Like, it was just too early, the wrong time, the ecosystem wasn’t built out for balance, the system components and services and everything else, the subsidy models were wrong. Because I actually hope that we can get some R&D to the future with cleantech.

      Andrew: We are getting cost declines with solar and wind. The price — the installed price, and then the price per unit of energy once they’re installed, is going down, you know, at a really attractive rate. So it’s not that we’re failing with these things. What I was trying to point out earlier is there are just some basic problems with that style of energy — especially because we’re not getting the battery revolution. And the battery nerds that I talk to say, “Look, there’s an energy density limit here.” So you’re pushing up against some physics. And it’s not that we can’t do anything about it, or that we should stop research. Of course, we should continue that going. But you got to backstop it with something.

      Sonal: With some portfolio. Right?

      Andrew: And that something, in my view, should also be clean. It should be nuclear. And then let’s let the battle rage for which is the cake and which is the icing. I kind of think nuclear is going to be the cake. And we’ll have a little solar and wind icing if we get it right. But maybe I’ll be wrong about that. Well, I just don’t want us to keep putting, you know, huge amounts of carbon in the air to generate electricity. We don’t need to do that.

      Marc: So, this is where — I don’t think environmentalism, for the most, part is actually about the environment. I think it’s about something else. And the reason I say that is because, exactly to Andrew’s point, I think we actually have the answers. I think we have the answers, and I think they’re nuclear — which is just, like, in practice an incredibly safe technology, contrary to what everybody believes.

      Andrew: Plus one to that.

      Marc: And then, I think, look — like, it goes back to the tariff thing. Like, let the Chinese build solar panels. Let them ride the manufacturing cost curve down, and, like, buy their solar panels.

      Andrew: Plus one to that.

      Marc: Right. And we have two magic technologies. Like, we have the box that generates power by splitting the atom, and we have the sheet that converts sunlight for free. And both of those are, like, incredibly modern production techniques for nuclear and solar. Like, it would just be, like — spectacular what you could do. If you engineered new nuclear plants today from scratch — like, properly, with the technologies. Most of the functional nuclear plants in the West today are, like, on average they’re — are there any younger than, like, 30 or 40 years old? The average is gonna be over that?

      Andrew: I think that’s right. I don’t know when the last new one we built was, but it’s been a while.

      Marc: And so, if we took current technology and did that, there are some really amazing ideas of things that we can do. I don’t think this has anything to do with the environment.

      Sonal: Well, what I find hopeful, though, about what you just said is that we have the answers. And that’s really important. And so a lot of these things come down to market and other dynamics — regulatory, politics, all of that. So it’s not a technological limit, which I find very helpful.

      Andrew: It’s also not a policy mystery anymore. We have these essentially magic technologies, where we should be stepping on the accelerator with them super hard if we really wanted to clean up the planet and stop polluting it with greenhouse gases. If we wanted a policy toolkit to reduce carbon, we have it. It’s worked for other kinds of pollution in the past. Carbon is not mysterious. <Yep.> It’s just comparatively politically difficult. <Right.> Now, I think some parts of the world will be more clear-headed than others. And I hope somebody else will show us the way, and their evidence is going to become unignorable at some point.

      Sonal: I also just want to make one pitch for the iPhone moment in cleantech, which I know people think can be very much of a long shot. But I think a lot of technology waves do have their major iPhone moment, where there is the technological tip that then drives everyone else to make cheaper versions of that thing later on, once there is this desire and demand and pull and draw to have the thing. And I actually have to say one thing that I did find promising about Tesla, and their move into solar for the home — and battery as sort of this backdoor, this Trojan horse — that the car is a Trojan horse to actually powering your home — that is a very powerful idea. And over time, who knows where that can go, but…

      Marc: I will say one thing that’s just from a consumer psychology standpoint. You know, Elon making electric cars sexy?

      Sonal: Yeah, that changed the game.

      Marc: That’s a big deal.

      Sonal: It was way better than Leonardo DiCaprio driving a Prius, which is what I drive.

      Andrew: That is absolutely a big deal, and, Sonal, to what you said, I’m thrilled that there are people willing to make some pretty risky bets on things. On the technology and the innovation front, I agree with Marc — we have some magic bullets. I’m going to mix my metaphors. We need lots of other shots on goal, right? And the innovation and the entrepreneurship ecosystem are a way for us to get more shots on goal. Hallelujah.

      Sonal: Yeah, and I’ll just say one last thing on that. One of the things that I find really fascinating is that there is this phase with a lot of technologies where there is that very down moment, where things go down, it seems like it’s dead. And in fact, the thing is being built out under the very surface, and you don’t realize that’s happening. And so to me, the death of the cleantech boom is actually promising because — Marc, you alluded to this —but it did fund — Elon Musk rode those subsidies to fund Tesla in the early days. And so who knows what can happen next?

      Andrew: I still think there’s a big place for government R&D. Again, more shots on goal, more attention to this — crazy ideas. And the reason Paul Romer won the Nobel Prize last year was he said, “Economies grow on ideas. Human Capital is the gating factor for increasing our growth and prosperity.” Let’s get more human capital out there.

      Sonal: Well, Andy, thank you so much for joining the “a16z Podcast.” Your new book, out October 8 — “More from Less: The Surprising Story of How We Learned to Prosper Using Fewer Resources and What Happens Next.” Thank you for joining “a16z Podcast.”

      Andrew: Sonal and Marc, thank you for having me. This has been a blast.

      Marc: Thanks, Andy.

      • Andrew McAfee

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Opioid Crisis

      Jorge Conde, Vijay Pande, and Sonal Chokshi

      This week we do a short but deep dive on the opioid crisis, given the data around where and who was behind the manufacturing and distribution of specific opioids:

      • How do opioids work, why these drugs?
      • Who’s to blame?
      • What are other directions for managing pain — and where could tech come in, even with the broader social, cultural, and structural context involved?

      Our a16z experts in this episode are a16z bio general partners Jorge Conde and Vijay Pande, in conversation with host Sonal Chokshi.

      Show Notes

      • Historical background on the opioid crisis, and current legal cases involving manufacturers [0:00]
      • How opioids interact with the brain and why they can be addictive [3:09]
      • Systemic reasons that lead to excessive opioid prescriptions, and who is to blame for the crisis [6:13]
      • Possible solutions for manufacturers, distributors, regulators, pharmacies, and physicians [8:30], and the need for better prescription management [13:16]
      • New technologies, including VR therapeutics, that may reduce the need for opioids [15:08], and a better understanding of addiction [16:35]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and I’m here today with the fourth episode of our new short-form new show, “16 Minutes,” where we cover recent headlines the a16z way, offering expert takes on the trends involved and more. You can follow the show in its own feed in your favorite podcast player app.

      Our other episodes cover multiple news items and topics, but this week we’re doing two separate, but short, deep dives connected to recent headlines. One on e-sports gaming and the future of entertainment, which you can find in this feed or at a16z.com/16minutes and this episode, which is on a sad but important topic, the opiate crisis.

      Just to quickly sum up, the issues of the opiate crisis have been around for years, which is this prescription opioid epidemic that resulted in nearly 100,000 deaths from 2005 to 2012. And what makes it even sadder is that it disproportionately affected people from regions that are underserved economically — for instance, native American tribal regions, towns in West Virginia, and so on. For what opioids are, as a reminder, remember the word opium — they’re a class of drugs that include heroin, fentanyl, pain relievers like Oxycontin, Vicodin, Codeine, morphine, and most of those are pain relievers that are legal and available by prescription.

      This crisis has been around for years, but here’s the news — the Washington Post and the publisher of the Charleston Gazette-Mail, which is a West Virginia paper — one of the regions that’s most impacted by this crisis — waged a year-long legal battle and won a court order for access to the drug enforcement administration’s database, which is this automation of reports and consolidated orders. It’s the ARCOS database. And basically, the Washington Post’s work helps visualize how much specific drugs went to individual states and counties, and who the top distributors, manufacturers, and pharmacies that were involved.

      And according to the Post’s high-level findings, just three companies manufactured about 88% of the pills, and just 6 companies distributed 75% of them. And over the past couple of weeks, a number of lawsuits have been filed as a result of those findings. Arizona just filed a case against the maker of Oxycontin. Unusually, they did it directly at the Supreme court level, while towns and cities are suing pharmacies like Walmart, CVS, and Walgreens. In fact, nearly 2,000 cases have been brought as reported by the New York Times, and their headline for that story by the way, was so perfect and so starkly sad — “3,271 pill bottles, a town of 2,831.”

      So, that’s a high-level summary of what’s going on, what’s in the news. I’d like to now welcome a16z Bio general partners Jorge Conde and Vijay Pande to talk about their views on this from their vantage point. Welcome, guys.

      Jorge: Thank you.

      Sonal: So, one bit of color from that New York Times story that is just so vivid and heartbreaking — one County in Ohio resorted to a mobile morgue just to handle all the corpses from people who died from overdoses, which is so sad. And as with all such things, science and technology does not live in a vacuum and plays out against a broader constructional context. So, I want to acknowledge that we’re going to be focusing on a specific angle, but really this is a huge problem on so many different levels. So, first of all, can you just quickly summarize the crisis from your point of view? Why opioid? What’s going on here?

      How opioids interact with the body

      Jorge: Well, first of all, opioids, as you said, are opium-based drugs, and it’s probably worth a moment to talk about kind of how they work and why they’re such a problem. <Yes.> Opioids basically target a receptor class within cells called the opioid receptors. And there’s three main classes, and the three main classes all have slightly different functions. And by the way, as we learn more biology — but I think identified another 15 or 20 subclasses of these things.

      So, the biology, as you can imagine, is complex, but essentially what happens with an opioid is that it targets one or usually many of these receptors and that has the pain-numbing or painkilling effect. It also hits some of our, you know, essentially our pleasure-seeking centers. So, it has the addictive effect…

      Sonal: Hence, the addiction.

      Jorge: And by the way, it also hits other important receptors that are necessary for, sort of, our physiological function. Most notably, one of the subclasses of receptors is responsible for sending the signal to your brain that you need to breathe.

      Sonal: Whoa, I had no idea.

      Jorge: Yeah, no. A lot of people that overdose and die from opioids, <Oh…> really what they die from is forgetting to breathe. And in fact, like the recovery drug Naloxone, it basically competes for the drug off that receptor so the person actually comes back <Wow.> and remembers to breathe. So the drug itself is incredibly powerful, and I think one of the important things to remember is that addiction isn’t weakness. It’s not lack of willpower. It’s actually a weakness of the biology that the opioids target.

      In fact, I remember when I was in graduate school, I took a pharmacology class, and the lecturer at the beginning said, you know, if I took this classroom of very accomplished, intelligent, driven, responsible graduate students, medical students, and gave everyone a dose of heroin, a significant proportion of a significant majority of this class would be hopeless addicts tomorrow.

      So, a big part of the problem here is that this is a very, very powerful class of drugs. And what’s really tricky about opioids is that a more powerful drug is not necessarily a better drug.

      Sonal: First of all, thank you for acknowledging that this is not necessarily a choice that people make. That’s really important, that it’s biology, but you also mentioned heroin in that example. And that one is an illegal one, which is of course a class of opioid, but most of these are prescribed. So I’m curious how that plays out.

      Jorge: First of all, biology is a very dynamic system. And so if you take a drug, any drug, really, you start to — well, you tend to develop tolerance for it over time. And it can happen via various mechanisms, but one of the mechanisms that’s believed to be the case in opioids is that as you, essentially, take the drug, your receptors essentially become accustomed to it, and so it actually changes the dynamic of the receptors.

      And people describe it as, you know, if you take opioids for a long time, you are quite literally changing your brain. And so the result of that is, if you’re taking a drug — and especially for relieving pain — you may need more and more of that drug to relieve pain. If that particular opioid also happens to target or hit one of the receptors associated with what’s linked to addiction, over time you’re going to seek more and more of it. So it just becomes a truly biological dependence at the cellular level for these drugs.

      Origins of the crisis

      Vijay: You know, it’s important to consider why patients are getting these in the first place.

      Sonal: Right. Quite honestly, if that — if this is, kind of, by the biology, is that you become more addicted as you take it, why are they getting it?

      Vijay: And there’s two reasons, which is somewhat of a shift. So one reason is that there’s been a recent shift in policy that essentially no pain is acceptable. So, you know, they often ask you if you’re in the ER or something, like, what’s your pain from 0 to 10. And it’s not that everyone’s saying 10 and then they get fentanyl, it’s the belief that no pain is acceptable. And this is actually very much an American thing. In other cultures, you know, you may be under extreme pain, but you’ll get tea, or you’ll get maybe Tylenol or something very different, and it’s just understood that you have to sit with the pain.

      The second, man, it’s just the healthcare system now is so strained that if let’s say you have major back pain and you should maybe be seeing physical therapy or maybe you should be seeing a doctor for musculoskeletal, it may take you four weeks, six weeks to see that doctor.

      Sonal: It takes time to see an expert.

      Vijay: Yeah. But you could get the prescription immediately.

      Sonal: So some of this is tied to health care access.

      Vijay: Yeah. But then, you know, it puts them in this bind where they really should be getting physical therapy or something like that, and they are on this path. The third thing is that often the alternatives are harder short-term, like physical therapy is a lot of pain. And so this is just, it’s available, it’s thrown on you by a doctor and it’s easy. You put those things together, that’s the match on — that lights the fire.

      Sonal: So, this is very helpful for helping break down the biology and the science behind this. It plays out against broader structural factors, cultural factors, political factors. This is a really big, important topic. And I have to ask, who’s to blame? Like, the interesting thing is that the news — there’s all the lawsuits happening to these pharmacies. And now the pharmacies and distributors, they’re coming back and saying, well, what about the impact of doctors and criminal drug dealers? Politicians — they are the ones who are trying to hide the database. There’s so many different players going around here. I want you guys to tell me, like, who’s to blame.

      Jorge: I mean, embedded in the question is part of the answer. I think really what we have is a massive systemic failure. I mean, you talk about manufacturers, you talk about distributors, you talk about pharmacies, you talk about prescribing physicians, and ultimately, you talk about patients and their families and their caregivers and sort of the communities that support them. And then you also talk about the politicians, you know, the public health agencies.

      Correcting systemic failures

      I think the systemic failure here is pretty broad. So, we can start from the very beginning, which is — we do need better opioids. Now we do need better painkilling drugs. We need, as Vijay mentioned, to be more thoughtful about how and when we intervene with pharmacologic drugs for pain. One of the things that you can’t do with an opioid is you can try to design something that is only hitting the right receptor.

      Sonal: This goes back to your earlier point about there being 15 types of receptors that are now being discovered. You can get more and more precise.

      Jorge: Exactly. So, now that we can engineer cells and we can work with cells, we can find very precise ways to understand what molecules are interacting with what parts of the cell, and design molecules that are hitting just the right notes that we’re going to be, you know, more targeted. So, there is the potential for a better opioid. By the way, to date, most of the attempts to improve it have been to address the, you know, ways to not tamper with it, so you can’t overdose on it. But the reality is you can get a better molecule if we understand what’s driving the biology. So, that’s the first step on the manufacturing side. The second one is, yes, the distributors and the pharmacies. I mean, the biggest problem is that this is a very ad hoc, disjointed system that we have here in the United States.

      Sonal: Like the healthcare system.

      Jorge: The healthcare system. And so I think a lot of what you’re relying on in terms of the crisis is that there aren’t really the checks and balances and the alert systems that you would — one would expect in place that doesn’t require sort of a human being to say this, you know, employees flag one particular shipment, but that one particular shipment or that one particular prescription obviously doesn’t catch the systemic problem as it’s evolving. And so you’re really missing, you know, the forest for the tree.

      Sonal: Is that a place that tech can help?

      Jorge: It’s an absolute place that tech can help because, I mean, first of all, a lot of this is by requirement that you have to inform the public health agencies if there is the suspected overuse of a controlled substance. And so, instead of requiring people to voluntarily do that, you could deploy technology-based systems that essentially do that automatically.

      Sonal: In fact, one of the quotes in the New York Times article came from a Walgreens official who said that he was the one who was tasked with monitoring the orders — said his department “was not equipped for that work.” I mean, that seems like an obvious place that tech could literally do what you’re describing.

      Jorge: And it’s a place that tech could do it far better.

      Sonal: Exactly. No, that makes great sense.

      Vijay: You have to understand, I mean, what’s going on in a lot of these places, it’s Post-its, fax machines. It’s something where, you know, the things that we take for granted that on, sort of, just coordinate our daily lives — could be put in here and could really have a significant impact.

      Sonal: Okay. So, let’s go back to the systemic players and failures. We have manufacturers, distributors — let’s continue breaking each one of those down.

      Jorge: On the manufacturer side, there’s really two issues here. One is we do need better drugs, as we talked about. And number two — and I think this is a very important point — is, you know, a lot of times in companies as they’re commercializing drugs, obviously, the goal is to grow revenue and that can, you know, sometimes create perverse incentives to drive usage where perhaps there shouldn’t be usage. And I’m not saying that’s necessarily the case here, but that’s something that I’ve seen happen, unfortunately, across the industry over time.

      The second issue is the distributors. The distributors are obviously responsible for moving product through the channel. They, of course, have incentive to move more product through the channel. And so, you know, if there are no controls in place, if the right tensions aren’t there between how things are prescribed, or how things are reordered, or how things are pulled through the system, that could also create a perverse incentive from a distributor standpoint.

      And I think you show some of the concentration of what happened in the case of the — of this particular episode of the opioid crisis, as you’ve laid it out. So we do need checks against the distributors as well. When you get to the pharmacy, the pharmacy is where the rubber meets the road, right? These are where the prescriptions are getting picked up, or getting shipped to, at least. And so, if you don’t have, you know, a manual control system there — I actually think that the biggest problem is just lack of an alert system. If I go in today to pick up a prescription, there is no real system that would raise flags, at least not efficiently at the system-wide level. It tends to happen very episodically, as this story itself has shown.

      And then, finally, there’s the physician prescription challenge. Because patients are in pain, the physician may not want them to tolerate pain, so may be more likely to offer this to offer immediate relief. Two, you get to the point where if you have to wait weeks and weeks and weeks to see a specialist or to get therapy, or to get treatment, this is a fast fix, short-term solution that eventually might become a longer-term problem, obviously, as addiction becomes an issue.

      And the third one is — these are all related points, but physicians, for the most part, don’t have the right control systems to do really effective medication management. So my treatment of you is very episodic. I come — I see you, you describe pain, I will prescribe something. I may look in the notes and go back and see what had happened in the past, but I’m not really following this day to day.

      Better prescription management

      Sonal: And this by the way, applies across all health problems, not just pain and addiction. <crosstalk>

      Jorge: All health problems. Medication management, medication reconciliation is a massive problem across the entire healthcare system. The particular challenge here, of course, is that this is the one area where a more powerful drug leads to more usage rather than less usage. And that’s what makes it so difficult when you can’t reconcile, you know, patient usage is happening over time.

      Vijay: And there’s ways that we could work within the existing system. Like, one thing you could imagine is a PBM that is more involved…

      Jorge: Pharmacy Benefit Manager.

      Vijay: Yeah, Pharmacy Benefit Manager, that’s more involved with clinical care, where they’re just — they’re not the doctors, but at least they’re better interfacing with the doctors such that you can at least have sanity checks. Like, there’s no reason why a patient would need this. And this way you can’t shop around to multiple pharmacies because you’ve got the same PBM, and it is that layer.

      And I think as you start to get smarter PBMs, these problems would be very naturally addressed. Not just for the opioid crisis, but it would be true for patients that have sometimes two or three drugs to treat the same condition, or three drugs that are actually gonna interfere with each other. Those are sometimes very difficult because in the medical system, you’ve got the endocrinologist and the cardiologist and the psychiatrist each prescribing without really any coordination.

      Jorge: And you know, to that exact point, we have a problem in the healthcare system of getting things de-prescribed.

      Sonal: What do you mean by that, de-prescribed?

      Jorge: Well, patients might be taking a medication for an acute condition. And, you know, I saw the physician and the physician told me to take this medicine for condition X…

      Sonal: You got a — you broke your arm and you need Vicodin…

      Jorge: Or you may have, you know, you may have a heart condition that has a — that’s going through an acute episode. Any number of things that I’m on 10 different medications, it could be that the condition for which this one drug was given to me has since been alleviated, has since been addressed…

      Sonal: But that information doesn’t get plugged back into the system to close that loop…

      Jorge: Yeah, and I don’t know how to stop taking it, so I might be taking a medication that I don’t need for a long period of time. And if somebody doesn’t do the reconciliation that Vijay just described, I could be on many medications that not only interfere with each other, which is a problem, but that I may not even need, which is a different problem.

      Possible tech solutions

      Sonal: So, that kind of addresses it at the, sort of, structural, logistical level of the healthcare system. Now, back to the point you brought up about the biology and some of the pain management. I mean, there’s obviously alternatives like TENS devices, and all kinds of things that could potentially scale in the future to address pain, but now let’s go to what the fixes are. Obviously, there’s social societal things that need to be addressed, but what can tech and science help with here? Are there any other future directions from your vantage point on the bio side? Clearly, there’s technology to address the transparency, the PBMs, the pharmacy Benefit Managers, closing the loop, everything from manufacturer distribution to prescription. What are some of the other things, what are some of the interesting directions you see to help address this?

      Jorge: Well, there are efforts to develop digital therapeutics, VR-type applications…

      Sonal: Right. And by the way, digital therapeutics as in, like, apps and things like technology that can actually be — act as if a drug in helping people to better outcomes?

      Jorge: Exactly. Maybe that can help you — get you into a state of mind that might help alleviate the pain, right? So, you know, if you can find different ways to address the pain issue, whether it’s, you know, physical therapies or something maybe novel like, you know, quite literally having a VR, virtual reality-type experience, or having an application on your phone that helps you meditate or calm down that might address some of the pain issues, you may not be as dependent on getting on the opioids in the first place.

      Sonal: I’ve read a ton of papers, actually, that VR has already proven to be effective in helping with PTSD — post-traumatic stress disorders — with veterans coming back from wars, or, you know, people who are suffering severe depression. It’s just really amazing that it can help.

      Societal influences on addiction

      Vijay: Well, you know, I think often we are worrying about the consequences without thinking about the source. Jorge made a great point about how addiction is a natural consequence. There are other recent studies that talk about, sort of, a little deeper about why this is so.

      So, the famous one is called the “Rat Park” study, where they actually had rats in a cage, which is kind of like jail, and given the choice between food or opioid, they’ll take the opioid until eventually, they kill themselves. But if you give them access to Rat Park where they can play and be social and, sort of, just live their normal happy lives, then actually giving the same choice they would choose the food and not the opioid.

      We know that social determinants are a key part of healthcare — it’s just not wrapped into a fee-for-service kind of system, where no one’s job is to take care of these things. But if we could take care of the root causes of this, which are beyond just about prescribing drugs but about thinking about healthcare as a societal issue, I think then we can actually really have a huge effort.

      Jorge: And there are several efforts ongoing now to use technology to help try to pull in all of those stakeholders in the community that can have such a big impact on some of these social determinants of health. Without that, this is another example of a fragmented system. A very analog system is — you’re doing this with call sheets, and coming up with referral names, and calling and trying to get appointments, and, you know, inbound visits and things like that.

      And it’s all necessary because this requires human intervention, but the coordination shouldn’t also be human. So I think technology has an opportunity here to have a massive impact on how we coordinate all of these stakeholders. So the people that may be more susceptible given some of these social determinants are more supported.

      Sonal: Right. And it just goes back to the bottom line for me, though, which is [that] technology is social, and it lives in a broader cultural context that clearly plays. Well, thank you so much, Jorge and Vijay, for joining the a16z podcast “16 Minutes.”

      Vijay: Thank you.

      Jorge: Thanks for having us.

      • Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      What Time Is It? From Technical to Product to Sales CEO

      David Ulevitch and Sonal Chokshi

      Since the startup (and founder) journey doesn’t go neatly linear from technical to product to sales, tightening one knob (whether engineering or marketing or pricing & packaging) creates slack in one of the other knobs, which demands turning to yet another knob. So how do you know what knob to focus on and when? How do you build the right team for the right play and at the right time?

      It all depends on “What time is it”: where are you on the journey, and where do you want to go… In this episode of the a16z Podcast, general partner David Ulevitch (in conversation with Sonal Chokshi) shares hard-earned lessons on these top-of-mind questions for founders; as well as advice on other tricky topics, such as pricing and packaging, balancing between product visionary vs. product manager, how to manage your own time (and psychology!) as your company grows, and more. Much of this is based on his own up-and-down, inside-outside, big-small-big-small, long journey as CEO (and CTO) for the company he co-founded, OpenDNS.

      The company was later acquired by Cisco after it pivoted from consumer to enterprise. Speaking of, what are the latest shifts and nuances in selling and buying enterprise products, beyond the phrase “consumerization of enterprise”? Or beyond the cliché of “design thinking” — how does one go beyond user experience and beyond things like fun gifs (which are pronounced, ahem, “jifs”) to focusing on the whole customer experience, and earning the right to be complicated? All this and more in this episode… plus the magic 5 words that will help any CEO (and anyone, really).

      Show Notes

      • Advice for founders, including the skills needed at various stages of growth [1:51]
      • Improving customer interactions [6:13] and the importance of creating value early [8:12]
      • How Slack moved into companies from the bottom up, and making software useful [11:24]
      • Discussion of packaging [14:15] and how to make packaging and pricing decisions [18:11]
      • Adding leadership as a company grows [19:57], including a product manager [25:30]
      • The origin story of OpenDNS [30:44] and how it pivoted to a new business model [33:57]
      • David’s shifts in roles within OpenDNS, and a discussion of the role of a CEO [37:22]
      • Learning to listen [41:42] and David’s background in anthropology [43:57]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and I’m here today with David Ulevitch, one of our new general partners who covers all things enterprise. But honestly: that can mean so many different things to different people, so we briefly discuss what “enterprise” products really mean today, for entrepreneurs, companies, and users — especially given the latest shifts driving SaaS beyond the cliché of “consumerization of the enterprise”.

      We also cover specific advice on the topics of:

      • pricing and packaging;
      • how to balance being a product visionary with being a product manager;
      • when and how to scale out and hire your leadership team, and how do you know that’s working or not; plus
      • how to best manage your own time — and your own psychology! — as a leader while doing all this.

      For context: David founded OpenDNS, where he went through a rough period in going from CEO to CTO in 2009 and then back to CEO again, in a company that itself pivoted, from consumer to enterprise. We discuss how did he make a comeback; it’s not like he changed instantly overnight, so what and how were the lessons learned? In 2015, Cisco later acquired OpenDNS, and David then ran their security business, where he also led the acquisition of three companies before coming here. So he’s seen startups from all sides: from being acquired to being the acquiror; from small to big to small to big again; from on the inside and from on the outside.

      But the real theme of this episode is the journey many founders make — from technical to product to sales CEO — and while we end with the story of OpenDNS and the most important lessons learned there, we begin with what is the one piece of advice David has for founders?

      Advice for founders

      David: So I think that there’s not one — you know, I’m not a big fan of platitudes, where you just say one thing and that applies to everyone — because there’s never one thing that makes the difference between success and failure.

      As a founder, you’re generally at different stages of the company-building journey. Sometimes you’re a technical CEO, trying to build a product, to make sure it actually is feasible; then you are constantly in the market with customers, doing customer discovery, making sure that you are solving the problems you’re trying to solve. You end up becoming sort of a product CEO, making sure that you have product-market fit; then you end up becoming a sales CEO, on the enterprise side, where you’re trying to generate revenue and figure out how you can go acquire more customers. And then if that works out, you become this really general sort of manager or go-to-market CEO, and you’re thinking about how do you scale and accelerate a business.

      And so, I like to think about those as different journeys, where there’s the right decisions for the right time — and really trying to help founders understand, what time is it, and where are they in that journey, and where do they want to go?

      Sonal: Is it possible to be all four at once, or is it really tied to the stage of the company?

      David: Generally, you want to be more focused than less focused. The reason you’re probably also not all those things at the same time, is a company is a set of these interconnected little knobs, and you can never just optimize one and then forget it and not come back to it. You end up going to another knob. If you fixed pricing  and packaging, then–

      Sonal: It’s like a control panel.

      David: Yeah, it’s like a control panel. Then you’re going to move to demand-gen (demand generation) to try to increase the top of the funnel. And if you fix that, then you want to make sure that your SDRs (sales development representatives) and salespeople are converting marketing leads into qualified leads. Then you’re closing, then you’re doing customer success. And once you tighten one of those knobs, it just creates slack in one of the other knobs. You might switch those hats from time to time, but I think you’re rarely going to be wearing more than one of those hats at the same time.

      Sonal: So I have to ask, since we’re going with this theme of let’s not do the platitudes — a lot of people say “it’s about the customer,” and “the customer journey,” and “understanding the customer.” Honestly, when I hear that I’m like, they [customers] can pull you in a million different directions, you don’t even know what to do. Especially if you’re a technical founder, you don’t know who to sell to; there’s a bit of a chicken-egg. So how do you figure out how to sell to the right customer?

      David: It’s okay for a company in the early stages not to know exactly who they want to go after, but they do have to understand the consequences of the customers that they’re targeting.

      I think we’re living, today, in one of the best times to be an enterprise software startup. And, to me, one of the reasons is because so many companies today are SaaS and subscription-software companies where there’s a recurring revenue component — it’s better for the customer. Because they know that the customer experience is gonna be good… or they’ll stop paying for the subscription, right?

      Sonal: It’s a repeat business; it’s not a one-time sale.

      David: It’s a repeat business, right. I never think about the first sale when I look at a business, I always think about, what are you going to do in year two, year three, to make sure you renew the account, to grow the account? It’s actually a way of peeling back the onion to figure out how confident are you on where your product is today?

      Because if you say, “Oh, we’re only doing three-year contracts”, well, is that because it’s really hard to implement and tough to get customers onboarded and so you need runway to get them happy? Or is it because you think you’re overselling your capabilities set, and you just don’t want a customer to figure it out within a year and not renew.

      But if the startup I’m talking to says, “Oh, well, we did a couple three-year contracts, but we realized that we were priced really low, and so now, if a customer wants a three-year contract, we’re actually going to charge them more on the out-years.” Well, that tells me your product is really good, and getting better.

      Sonal: That’s fascinating.

      David: Another way to think about that is that what used to just be a product experience is now much more of a customer lifecycle experience: It starts before you sell, building evangelists; then there’s the onboarding part; then there’s making sure the customer is really happy. How do you market to your existing customers to make sure they’re getting full utilization of your product?

      That customer lifecycle makes it much easier for a startup that’s getting started to really identify who is the target customer. And then thinking about does that actually map to the business I want to build: Is it the big Fortune 500, the Global 5000, is it SMBs (small and mid-sized businesses)? Because it has all these downstream effects.

      It’s okay for a company in the early stages not to know exactly who they want to go after, but they do have to understand the consequences of the customers that they’re targeting

      A lot of startups will come in here — and in my first six months here, I’ve now met with over 200 companies — and a lot of them have this ambition to go after SMBs. And one of the cool things about SaaS is that SaaS can take something that in the olden days of enterprise computing, you’d have to buy the biggest server, the biggest box, to get the best solution. But with cloud and with SaaS applications, you can now have the power to get the massively great CRM system, the massively great HR system…

      Sonal: I like to think about it as SaaS is very democratizing, actually.

      David: SaaS is totally democratizing.

      Sonal: Because it enables smaller- and medium-sized companies to have access to big company resources. They don’t have the in-house engineers, but they can essentially “as-a-service it in” to their company.

      David: Absolutely. Small companies have unlimited compute. They have unlimited storage. They have unlimited bandwidth now.

      Creating value for customers

      So when I meet with startups, they often want to ambitiously and altruistically go satisfy this pain for SMBs. But it turns out that the reality is, if you want to charge a higher price point, if you want to pay an expensive sales force, then you’re going to realize that your average deal sizes have to be higher. If I really want to go after a target market where the price point is going to be lower, then I have to think about bottoms-up sales, about self-serve offerings.

      I love when I see startups that think not just about who they want to go after, but then they build that into their whole customer experience model — marketing programs, pricing and packaging, renewals, sales — and the whole business model.

      I mean, nowadays, think about how many emails you get where it reminds you of a new feature that you may not even have known existed. For instance, I’m using an email product called Superhuman, and every week, I basically get an email from the team saying, “Did you know that you could use this functionality? If you press [command] ⌘ + I, it’ll automatically route someone to BCC in the reply. Or you just press ⌘ + C, it will copy the whole email; you don’t have to select it first.”

      Sonal: Actually, I saw this awesome tweet from Patrick Collison, the CEO of Stripe, where he said, I feel like Rahul Vohra, the CEO of Superhuman, is essentially inventing new user-experience interaction paradigms that will eventually cascade into other products. Much like Steve Jobs did with letting us learn new behaviors like how to touch a phone.

      David: Patrick’s tweet was right on. In Rahul’s weekly marketing email to existing users, it helps teach me these new things that they’ve unlocked. They become very intuitive, but you still have to learn about them.

      One of the best things about building a company today is it’s easier than ever to get close to customers, to constantly get iterative, real-time feedback: both from an analytical standpoint and from customer surveys, NPS scores, all these kinds of things. You have all this telemetry through SaaS products where you actually see how people use your product.

      We’re living, today, in one of the best times to be an enterprise software startup… You have all this telemetry through SaaS, where you actually see how people use your product

      But the second thing that’s happened is, people talk about this bottoms-up SaaS motion, but it’s not always just that. It’s really about making sure they understand that there are evangelists in the company that you have to win over, before you’re going to get sign off from the CFO, the CIO, the CSO, whoever he or she is that makes a decision — you’re going to have to get some champions underneath that person to be your evangelist internally.

      Sonal: So this is a little counterintuitive too, though, because the other piece of advice I’ve often heard from folks is that the #1 mistake a lot of consumerization-of-enterprise type of founders make is that they go TOO heavy on bottoms up, to the point of ignoring the importance of top-down sales. What’s your view on that?

      David: I like to frame it a little bit differently. When people talk about the consumerization of enterprise, enterprise customers today are being bombarded by so many different vendors, their attention span is so limited. I think when people say consumer, they mean “easy”. They don’t really mean “consumer”.

      The value proposition that I’m hearing when I talk to customers is that the time-to-value needs to be short. It’s actually two parts: The first is, I want value almost immediately, sometimes even before I pay for it. I want a trial, or I want to get up and running on my own, and then I’ll talk to a sales person. The time-to-value can either be t-minus zero days (negative time), or it has to be very short from hours to days.

      I always encourage founders to think about the first hour, the first day experience, the first week experience, the first month experience.

      The second part is that enterprise software can get quite complex. Zoom is in the news because they went public, and Zoom’s a great example of something where they earned the right to be more complicated.

      Sonal: Wait, let’s pause on that for a minute: Earn the right to be more complicated.

      David: So this goes hand-in-hand with a short time-to-value. The short time-to-value gets you in the door. We know that you and I could download Zoom on our phones and be in a video conference call, but now we’re like, “Well, wait a minute, maybe all of our conference rooms should have Zoom. Maybe we should integrate Zoom with our Google Calendar.” You’ve earned the right to do that complexity because you’ve already proven so much value. And not only that, the value you get by doing the integration with G-Suite, or by adding some cameras to your conference room so that you can have Zoom-rooms — that complexity is commensurate with the value you’re getting.

      When people say [that] “consumerization of enterprise” means it has to be easy or simple, that’s not quite what it is. To me, it’s two things: It’s a short time-to-value; and then the complexity curve is commensurate with the value proposition.

      Sonal: So then I wanna ask you more about what needs to go into that “time-to-value”. So, let’s be a little bit more specific.

      I mean, I get the point that it’s incredibly competitive, so you’ve got to differentiate fast and show the value. But what are the things that drive that? Is it a great… like, a cute little JIF-y, <chuckles> that, you know, that jumps out at you and makes cute — like, a Clippy-type of thing? I mean, what is it?

      “Sonal’s right, dammit” (image: Giphy)

      David: Did you say a JIF-y, like a GIF?

      Sonal: Ugh… are we gonna rumble?

      David: Wait, I know how we can do this. <laughs>

      Sonal: Are we gonna just stop and, like, end this?

      David: I can’t have you trolling me.

      Sonal: We can’t be… I am not trolling you! I’m one of the people who calls JIFs, not GIFs (I hate that). You know there’s a world of people that think they should be J!

      David: Hold on… I don’t know if I would have agreed to this podcast if I knew you called them JIFs.

      Sonal: It’s like nails scratching on a chalkboard. You’re literally right now in my ears. It’s like someone’s poking pins in it right now.

      image: Giphy

      David: How have you been so successful in your career this whole time calling them JIFs??

      Sonal: I actually feel like I kind of hate you right now, to be honest.

      David: Aw, this is amazing… <laughs> Oh, my goodness. <laughs>

      Sonal: You don’t have any friends that pronounce “GIF”, “JIF”?!!

      David: I don’t think so. Like, you’re it.

      Sonal: Chris Dixon, is he a friend of yours?

      David: He is.

      Sonal: So I’m outing him on the podcast, because I’m not gonna go down in this ship alone. He’s another one.

      David: You might have to edit that part out, because I don’t know if that can be out there… There’s, like, this dissonance in my brain, because you and him are so smart, but you also call it a JIF. <laughs>

      “David thinks he’s right” (image: Imgur)

      I don’t know what to do right now… <Sonal giggles> All right.

      Making products useful

      Sonal: Okay, so, Slack, for instance, they did a lot of really creative things. I remember I was at Wired, and the product that we used was HipChat. And the thing that eventually got me into Slack was the fact that you could do all these GIFs (whatever;) — <David: yup> you could do more fun things.

      David: And they have the integrations, other things could drive information into your Slack channel. And that was not something that happened to HipChat for a long time.

      Sonal: That’s right, like Google Documents and…

      David: That’s right, Dropbox files. Even automated updates, like, if you’re a developer, when somebody would do a push to production, it could notify people inside the Slack channel.

      Sonal: Right. But now that’s not a case where IT has to decide the integrations.

      David: That’s right. And they made it easy for individual users. As long as you could use Google Auth to authenticate, anybody could basically set up a Slack channel inside their organization.

      After a while, IT says, “Hey, wait a minute. We have all these teams that are chatting on this thing, they’re doing integrations, files are being shared. We need to have a little bit more visibility, a little bit more access control.” And for security and compliance reasons, it became an enterprise sale that went wall-to-wall. It’s now already entrenched in the organization. There’s already integrations happening with some of the developer tools and workflows, and at that point, they’ve earned the right to be more complicated.

      Sonal: I’ve noticed this resurgence — and I don’t know if it’s just like a zeitgeist thing or anecdotal evidence — of design-focused startups, precisely because of the thing you’re saying, because that’s one of the ways to instantly differentiate.

      David: “Design thinking” is sort of another way of saying…

      Sonal: Oh, I hate that phrase.

      David: I know.

      Sonal: Talk about the platitude-of-all-platitudes. That phrase drives me fucking up a wall.

      David: So here’s a better way to frame it. <Sonal: Okay> Cuz I also don’t like that phrase. <Sonal: Yes> Is it’s really about that extension of the product experience, and really taking that more holistic approach. It’s not just about the UI, it’s not even just about the user experience of a particular workflow, it’s about that whole customer experience.

      We’re actually entering a period of time where more and more people in the workforce are digital natives, and they want to be power users. You know, why isn’t there an equivalent Microsoft Excel on the web? Like, Google Sheets is not Excel. The current state of collaborative tools in SaaS apps is just so weak, and they don’t let you be a power user.

      What used to just be a product experience is now much more of a customer lifecycle experience

      Sonal: It’s also, I think, ignoring the realities of organizations today.

      David: Totally.

      Sonal: Which used to be so siloed. And now you have people collaborating cross-functionally in different ways…

      David: You could argue though, Google Docs did create a multiplayer mode where you could have collaborative editing, but it was just such a garbage experience from a functionality standpoint, that…

      Sonal: It was an afterthought. It wasn’t baked in natively. That’s, basically, my rule of thumb for all of this: If it’s an add-on, it’s not important.

      David: I would say Google G-Suite is an add-on too. Google should just shut down G-Suite altogether <chuckles>, even though the whole of Silicon Valley would go crazy. I mean, they’re a rounding error in their business, they’re a rounding error to productivity, versus what Microsoft has. I don’t think they’ll do that, but strategically, it’s just so unimportant for them.

      Sonal: Right.

      David: What I would say, though, is that I like software that is easy to use, that has that short time-to-value, but that also allows me to be a power user if I want to be.

      And in fact, as an investor, when I talk to companies, I always try to figure out what is their pricing and packaging strategy.

      Packaging and pricing

      Sonal: So tell me, what is packaging, actually?

      David: So, packaging is usually (I mean, it can be a bunch of things) — but to me, packaging is: What set of features are you going to put into an offering to a customer?

      I always try to think that you want to make it easy for your customer to give you money — like that is a foundational principle for me — packages are a way to do that. We’ve all been to the restaurant where the a la carte menu is all over the place, but sometimes restaurants just say, “Well, here’s the three options: Comes with one of these appetizers, you get this main course, and you get this dessert.” If you want to make things easier for people to give you money, generally, people come up with packages. And the friction is removed to becoming a buyer.

      In the SaaS world, sometimes there might be a tier that says, you’re going to get the full functionality of the product, but you’re not going to get archiving and logging and all this detailed reporting and analytics. It allows the company that maybe doesn’t want to spend as much or isn’t as big to get the full functionality of your product, but then there’s a hurdle. When I think about packaging, usually there’s a key product milestone that happens that forces somebody to jump to the next tier.

      Sonal: Interesting. What do you mean? Give me an example of that.

      David: Well, sign-on is a good one. Lots of SaaS offerings let you create accounts and use a product, but if you want to tie it to your Okta directory or some other directory service, you’re going to have to jump to a much more expensive tier. But, generally, the customers that have to jump to that tier are more enterprise companies. They have a directory service. They have a single sign-on service. They might want two-factor authentication with tokens. The security person in me doesn’t love that one being a tier because I always think you want all your customers to be secure. But there are other tiers, like compliance. If you’re in a regulated industry, you might not just be satisfied with 30 days of logging. You might need 365 days of logging. You might need to be able to export your logs to another data store.

      When people say consumerization of enterprise just means it has to be easy or simple, that’s not quite what it is. It’s two things: a short time-to-value, and the complexity curve is commensurate with the value proposition

      Sonal: So far, if I heard that as an entrepreneur, though, I would assume that all packages are tiered. Are there un-tiered packages where it’s just a different combo that’s kind of horizontal?

      David: You know, I don’t think I’ve seen that. Generally, it’s much more of a ladder — <Sonal: That’s what I was wondering> Where the next package includes everything in the previous package. And I think that, while there’s usually a number of features that get unlocked when you go to the next package, to me, there’s always one that has that forcing function.

      In fact, when I think of packaging, it’s often a way to segment your customer base. Because you’re going to say, we know SMB mid-market (under 1,000-employee companies), they’re going to be in this package. Everything we do, the product manager on that package is thinking about those features, thinking about our persona. And then the next package, the person is saying, wait a minute, I want to go after the 1,000 to 10,000-employee company, and this is what they need: This is how I communicate with them; this might be how I do webinars to them; this is how I’m going to do pricing that more fits their model. Maybe you can’t do a three-year contract if you’re on the low-end product.

      All these things are puts and takes that reflect where is the product, who is the customer you’re targeting, and then how do you want to market and create demand with that audience?

      With packaging, usually there’s a key product milestone that happens that forces somebody to jump to the next tier

      Sonal: Is there a balance or a rule of thumb — I’m sure it must vary by business — in what the ideal number of packages are; or, how many customer segments you should be trying to reach as a startup?

      David: Well, I think fewer is better because focus is key.

      Sonal: “Less is more”?

      David: Yeah, less is more. Time is always the most valuable currency in an individual’s life, in a company’s life. Aligning all that time behind the most important — putting more wood behind fewer arrows — is much more important.

      Generally: two packages, three packages. When you make it too complicated for the customer to figure it out, that creates friction to the sales cycle. <Right.> Now, with that said, one thing that startups often do is they share their pricing publicly on the site. And the engineer in all of us, the pragmatic person in all of us, we’re like, well, of course, we want to share pricing because as customers we hate not knowing the price. But as products get much more nuanced, and organizations are buyers, you actually don’t know what your pricing discovery looks like — you’re better off not sharing your pricing.

      Sonal: Okay.

      David: One way you know you have a great product is when your salespeople are the ones demanding you remove the pricing. Because that means that they’re telling you–

      Sonal: You can get more money.

      David: You can get more money! Maybe you’re a technical CEO who’s becoming a product CEO who’s becoming a sales CEO; if you’re listening, you’re going to be like, “Wait a minute. They’re telling me we’re leaving money on the table.” That’s generally a very strong signal.

      Sonal: I have a stage question on this, though, because if you think about the definition of “startup”, a startup, by definition, is a business under a high condition of uncertainty compared to a more established business — I wouldn’t even peg it to a particular size.

      Given that, a startup is an experiment; and the product, you can run multiple experiments at the same time… We’ve heard of the famous pivot — you know, the dreaded “P”-word — there’s all these different flavors of this. How do you run multiple experiments, and also strike a balance with focusing the pricing-and-packaging strategy?

      David: Well, that is the art of running a business…

      Sonal: Not a science!

      David: Yeah, and everything is multivariate. But you can — if you have smart people paying attention to the numbers, paying attention to the data, collecting the analytics, and giving yourself enough time to collect that data — the worst thing for a company to do is to make a decision and then not allow there to be enough time to collect the outcome of that decision, and understand the consequences of that decision, and then they make another decision.

      The question of how do you make decisions and run multiple experiments, I don’t think it’s that complicated — as long as you’re paying attention to: What are the outputs from those decisions that you should be looking for? And you should be looking at what’s changing across the business. We’re living in an era today of running companies where it’s much easier to collect and analyze data than it ever has been. You have data lakes where you can bring in product data, your CRM can tie into that product data. We’ve never had… we have BI tools now…

      Sonal: Business intelligence tools.

      David: That’s right. We have open-source business intelligence tools. We can actually run complex analytics and say, “Wait a minute, my West Coast territory is just doing so much better than my East Coast territory; what is the difference that’s pushing there? Is it because we actually are running more demand-gen campaigns on the West Coast, and the marketing team on the West Coast is separated, or is it just that the West Coast sales reps are better?”

      Sonal: You need to be able to tease apart those.

      David: You need to be able to tease those things apart. But it’s easier to get access to the data and analyze it quickly and avoid that analysis paralysis than it ever has been in the past.

      Narrowing focus and growing the team

      Sonal: So you know, a big part of this — so the big theme I’m hearing from you, is a lot of these things have intentionality, even if you don’t know the outcome. And that you can actually control that intentionality by being introspective, understanding your decision-making, understanding what works; that sounds great.

      Now, as the leader of the company, how do you, the CEO, figure out what to work on? And depending on what stage you’re at, this whole journey — from technical, to product, to sales, to go-to-market — that’s not necessarily perfectly linear, how do you figure this out?

      David: It’s not linear at all. I mean, sometimes in retrospect, we like to look and think that it was linear. <Sonal: Of course, right> I think that there’s different ways to figure out how do you prioritize your time, where do you spend your mental calories?

      Sonal: “Mental calories” — I love that phrase.

      David: Yeah, I mean that’s how I think about my day: “Where do I want to…” You only have so many mental calories.

      Sonal: That’s how I think about my day, too! I think of “nutrition density”. <David: That’s good> I have a phrase that I use for all my editing, which is “ROE (return on energy)”.

      David: Ooh, that’s good.

      Sonal: So I refuse to spend time on something that the output is gonna be vastly low-proportion outsized win to what the amount of work I put in <David: right> in terms of energy, creative. I have a whole framework for thinking about this stuff because I’m ridiculously productive on this front.

      David: Well, hopefully this podcast gets published <chuckles>, because then I’ll know that it had high ROE. <Sonal laughs> I think figuring out how you spend your mental calories is a really important question to ask, and sometimes, the act of asking that question itself is just part of the process of figuring out how to spend your time… and spend it wisely.

      And there’s different things that happen along different stages. I always look at what is the problem in the company — is it that we can’t get customers? And then figuring out who that right customer is.

      But as a company starts to mature — like, a lot of these companies get to this $2 million, $3 million in annual recurring revenue — that’s a huge milestone; very few companies ever get there. But, yet, it’s tiny when you should be doing $20 million, $30 million.

      If you aspire to be there, you can celebrate the milestone, but it’s clear that you have a long way to go to build an enduring, iconic company. At that point, though, you start to have a leadership team.

      Sonal: One of the biggest things that we see when we give technical founders advice, is they need to bring on a VP of Engineering, they need to bring on a Head of Sales. They keep resisting this thing because they’re attached to their early startup team. How do they figure out when to really– there’s a lot of religious advice and debates around this.

      David: I always go to the question of: What time is it? What is the priority? Are you trying to figure out product-market fit, or are you focusing on going to market? What time is it? Are you hiring salespeople and ramping up? Are you figuring out the customers are churning and you’ve got to go fix your product?

      Focus is so important. If you were to ask all your leaders and all the people in your organization — “What is the most important thing for our company right now?” — they should have an answer.

      Just one of the more tactical conversations that I have with leaders — especially when they’re a startup and they have this core founding team and then they’re thinking about scaling — and they say, “Well, you know, I have this engineering manager. He or she was with me from the beginning, and I think they’re doing a great job managing.” One of the things I highlight is that bringing in a world-class VP of engineering that could rock the boat, it could cause issues, but it’s not an indictment of your current engineering manager.

      You can celebrate the milestone, but you have a long way to go to build an enduring, iconic company. At that point you start to have a leadership team…

      Like that’s not what’s happening. Part of bringing on these high-performing leaders and these really well-respected leaders — that have a cult-like following with the people that have worked with them and for them before — is that they are going to help you accelerate your ability to recruit world-class talent. When you deliver that message to that person on your team who’s been there from the beginning and is doing a great job, that should resonate. It’s like, “Oh, wait a minute… We can get way better people, way faster? Yeah, let’s bring that person on.”

      Again, you have to be very careful about knowing what are the problems you’re trying to solve in the organization, but oftentimes — and I think VCs have a bad rep for this — they shove in somebody who’s way too senior <Sonal: Yes, exactly!> who comes from way too big of a company. You have to think about what is the right team I need for the right time.

      Sonal: I think Ben wrote about this in his book, actually, which is the mistake that people hire for the future instead of hiring for the thing you need now.

      David: This often comes up with VP-of-Sales hires, where somebody maybe has run a 10- or 20-person team, but you’re like, can they run a 500-person sales team? Well, you don’t have a 500-person sales-team problem! People often think about the executives they’re hiring and is this person going to do anything for four years, or five years, or six years; I think that’s not always the right question to ask.

      In fact, I had a board member once — Dave Strohm, who was a mentor to me, I think of him as the Yoda in my life — and he once said an expression that I’d never heard before, “horses for courses”. Have you ever heard of “horses for courses?”

      Sonal: No, I don’t even know what that is.

      David: It’s sort of an archaic expression. In horse-track racing, there’s like dirt courses, there’s grass courses.

      Sonal: Ohhh I get it!

      David: And you want to run the right horse for the right course.

      …There’s a bad part of this phrase too, though. <Sonal: Uh-oh> There’s a bad connotation, which is that, sometimes when horses, run their few races and then they’re finished, you don’t have–

      Sonal: They’ve “run their course”! That’s where it comes from, that expression!

      David: That’s right, they’ve run their course. And do you know what happens to horses that have run their course?

      Sonal: No, I don’t want to know. Are they turned into gelatin?

      David: Something like that.

      I always used to joke (it wasn’t very nice probably) — but I would joke with the VP of Sales I had at OpenDNS — that every quarter was his last quarter because he just constantly outperformed, and we always wondered when we hired him, is this guy going to scale? Now, he scaled wonderfully; he’s an incredible sales leader. He went from a 20-person sales team to, ultimately, a 200- or a 400-person sales team. Then once we got to Cisco, he did wonderfully. But we didn’t know when we hired him how far he’d get past 20 people.

      You gotta hire horses for courses.

      Sonal: The right team for the right play.

      David: And this is a good way of really figuring out, is my leadership team adding capacity for me? Are they helping me understand what’s happening in the business? Because, at a certain point as a CEO, you’re going to start to spend less time on engineering, less time on product. Ideally, you’re going to spend more time in the field with customers, with partners, with customer success. As you start to spend less time with any individual function, you’re going to need to have leaders in place that really are spending all their time really understanding closer to the metal with what is happening.

      Importance of product managers

      Sonal: I love that you said “close to the metal”, because that’s the exact phrase I use when I think of this: It’s like “bare-metal leadership”.

      David: Totally.

      Sonal: Because that’s actually the biggest challenge: As a product-oriented person — or a visionary for whatever the product is, in any field — how do you kind of keep that “close-to-the-metal insight”… yet, you can’t actually be close to the metal if you’re scaling.

      David: So this comes up a lot in startups, this idea that if you’re the product visionary, you’re the founder of the company, that means you are the product manager for the company. But at the same time, you need to scale an organization. And I think it’s important to differentiate the product manager from the product visionary. <Sonal: Oh, great>

      As the founder and CEO, you can always be the product visionary, but there is going to be a time where you’re not going to be able to spend hours of time with the engineers hearing how they’re working on a product, or how it’s technically going to work. You’re not going to spend hours and hours of time looking at all the NPS survey data or the customer support tickets that are coming in.

      And so, oftentimes, I’ll meet these startup CEOs who are like, “Oh, I can’t hire a product manager. I am the product manager.”

      Sonal: That’s a common thing for technical founders.

      David: Totally common! It feels like it’s your baby, you don’t want to let it go. But you’re only going to have five seconds a day to think about different decisions you make. And if your engineering team and the rest of the organization is constantly coming to you, you’re going to end up getting paralyzed. The worst thing for a product visionary is to make some decisions that they know were the wrong decisions — because they lack data, or they lack the time to be thoughtful about it — and then they start to undermine their own thinking about whether or not they even are a product visionary.

      When the reality is, just hire a product manager! You’re not offloading the product vision to that person, what you’re offloading is the day-to-day ground war of figuring out: What is customer support telling me? What is sales telling me? What is engineering telling me? What are customers telling me? Synthesizing, analyzing, prioritizing, sorting that data. Obviously, as the founder and visionary, you have the ultimate say, but you’re going to be armed with so much more insight/information, that your intuition — which plays a big role, too — is just going to be further enhanced.

      As a visionary, you’re going to have some special secret, some earned power, that you have over the lifetime of your experience, where you’re the domain expert in a problem set and know more about it.

      Sonal: Right, because you’ve gone through the idea maze. <David: That’s right> You’ve literally lived and breathed this thing; you’ve built the company, started it. You literally have it seeping out of your pores.

      David: That’s right. But how you build a product is not the same thing as having a vision for a product.

      Sonal: We had a recent podcast with Safi Bahcall, and he described how Steve Jobs had both the artists and the soldiers, and so not only did he have himself, but he had Tim Cook and Johnny Ive. And when you think about the story of the iPhone, the app store was actually a result of his team coming up with the point that, hey, you can’t just have only Apple apps on this, <David: That’s exactly right> if you want people to use this.

      David: Your product managers will come to you when they have conviction on something, and they have the data, and they have the view. You will then be able to make those bets. And nobody would say that he wasn’t a product visionary just because he didn’t come up with the app store.

      Sonal: On that note, though, just to probe on one bit — because I’ve always wondered about this — there is a tension between this idea (I hate this idea) of “the head,” and “the hand.” You can’t have one person be the “head” and the other person be the “hand.” How do you reconcile that bit? I guess what I’m asking is, how do you calibrate along this line of visionary to manager?

      You can differentiate the product ‘manager’ from the product ‘visionary’… how you build a product is not the same thing as having a vision for a product

      David: You want to know what you’re hiring for. Because there are product managers that are much more analytical, and there are product managers much more visionary. You might need different kinds of people at different kinds of times. I think you have to be self-aware and be really intellectually honest. Because, if you actually need someone who is more visionary, then you’re going to have to deal with the fact that you’re going to be going to battle and sitting in a room and duking it out over ideas.

      It leads to a secondary insight: Which is that, if you’re a CEO of a company and you do not trust that the information you’re getting from one of your leaders is what’s actually happening on the ground, that’s a tremendous problem.

      Sonal: That’s a huge red flag.

      David: Massive problem…

      Sonal: Fire and move on. Or it could be you, if you’re just not a trusting person.

      David: I think you have to work to resolve these things. You don’t just cut and move on immediately, but you have to work to understand, do they understand what’s happening, and are they able to communicate it to you and the rest of your leadership team?

      I always like to think of leadership teams, it’s not just, oh, the head of sales reports to the CEO; the head of marketing reports to the CEO — you have these like siloed, pair-wise conversations. The leadership team needs to be working together as a team and communicating with each other because, as a CEO, you don’t want to be interjecting and intervening in every conversation and every decision. You want to start to figure out, are they collaborating? Are they sharing each other’s experiences? Do they understand what’s happening in each other’s businesses? Are they meeting on their own?

      I think as a CEO, you actually want your leadership team to meet independent of the CEO.

      Sonal: That’s actually really interesting and counterintuitive.

      David: Yeah, I think it’s really important, and I think it does happen in a lot of high-performing teams very commonly, maybe not explicitly, but it happens. Then, obviously, in some places, you can do it explicitly — when it’s done in a productive and positive way, not because the CEO is a distraction. <Sonal laughs> Ideally, the CEO is out doing something that’s of high value to the company.

      But if you get to this place where you do not have confidence that you are getting the best information from your leaders, if you don’t resolve that, then you have to find someone who’s a better fit. When I talk to a CEO who’s having a tough time in the company, and they’re telling me what’s happening, I’m like, “Just tell me, do you really believe that that is what’s happening?” You either have to go deep (and as a CEO, you do get these occasional bullets where you can cause a little bit of organizational stress to go three levels deep and really figure it out) — and if you find out what’s happening is not what you were being told, you’ve got to make a change of leadership.

      You have to think about what is the right team I need for the right time

      By the way, I should just say that all my lessons about leaders and management, I have pretty much learned the hard way. So I’m just trying to help save other people from making the same mistakes I made.

      The story of OpenDNS

      Sonal: Yeah well speaking of that, let’s talk about your story. You’re the founder of a company called OpenDNS. First of all, what is OpenDNS?

      David: So OpenDNS is a cybersecurity service that delivers a faster and safer internet. We really innovated on a 25-year-old technology that used to be a cost center, that nobody wanted to innovate on. We proved that you can actually build a business on top of this thing that used to be free, if you make it better. Speed was one part, but then the other part was security.

      Let’s say you type in Zamazon.com, you’re meaning to go to Amazon.com, but that could be a phishing site trying to steal your credentials. We would say, wait a minute. We know that from our tens of millions of users, what you really meant to type in was Amazon.com, so we’re going to show you a page that says, “Hey, you typed in zamazon.com, we think it’s a fraudulent site, did you really mean to go to Amazon.com?” And that may help protect you from getting phished.

      It was the first third-party DNS provider. In fact, when we started the company, some of the greats of the internet told me: a) what I wanted to do was impossible; and b) even if it was possible, nobody would want it because guys would get it from an ISP (internet service provider).

      Sonal: Oh, my God. This reminds me of Marc with Netscape. One of my favorite stories is I saw these old forums that he was on when he was proposing a more of a graphical user interface.

      David: The image tag?

      Sonal: Exactly. And the thing that I thought was so funny is the people who are the established, kind of old fogies (for lack of a better phrase), they don’t like the change, ironically, even though they were very revolutionary at the time.

      So, you mentioned a 25-year-old technology… Why was that almost impossible to them?

      David: Think of the DNS like a phonebook. Except that what we wanted to do is not just give the same phonebook to everyone, we wanted to give a custom phonebook to every person. Let’s say you typed in Playboy.com. For some user over here, they may not want content filtering, so they want the answer for Playboy.com, but maybe for someone who has small kids at home, they want a different answer. Doing this at very high speed was thought to be impossible. <Sonal: That’s fantastic> But it turned out that it was possible, and we could do it faster than even if you had no preferences and settings.

      Sonal: I have to ask you, how old were you when you had the insight that you wanted to build OpenDNS?

      David: So, I had started a DNS company in college that did a different kind of DNS. And through that, I had gotten super interested in cybersecurity. I met an investor when I moved out to California who had asked me, basically, why I wasn’t doing more with my original company. Then, he and I ultimately came up with the idea for OpenDNS.

      That original business model that I worked on with him, it was an advertising-supported business model. We pivoted the business at the end of 2009 to having the people that use our service be the people that pay us for our service — it was just a much better alignment of interests, and that journey took a long time. By the time we pivoted the business, it really was a different business than when we started it. When we sold it in 2015 to Cisco, it was really a full-blown cybersecurity company.

      Sonal: Why did Cisco want it?

      David: If you looked at what happened in between 2007 and 2015. The iPhone came out.

      You had more and more people working from coffee shops that all had Wi-Fi; you had workers working from the road, people using mobile devices. So, installing like Norton Antivirus or McAfee Antivirus on your desktop was no longer sufficient security. And so our service, Open DNS, was cybersecurity delivered as a service. It happened intrinsically and as a part of your internet connection, so you didn’t have to have special software, you didn’t have to install an appliance or a piece of hardware. As people we’re working differently and the networks were becoming more ephemeral, Cisco (which is a major cybersecurity company, it’s actually the largest cybersecurity company) wants to evolve to match that shifting IT landscape.

      Sonal: You mentioned “The Pivot” a few times. Tell me about that because that’s such an overused — one of those platitude-y words, like, big P, little p, whatever — I know you mean it in actually what happened, but give me a little bit more texture around the pivot. What was that like?

      David: The best time I never want to have again. <Sonal chuckles> This might be a podcast unto its own, because there was an 11-month period where I wasn’t even CEO of the company. My original investor had fired me.

      Sonal: Oh, my god, I didn’t know that.

      David: I was CTO. I’d been demoted.

      Sonal: CTO is awesome, though. I think the CTO is the most powerful person in the company.

      David: Not when you wanna be CEO.

      Sonal: I guess that’s true.

      David: We pivoted the business in 2009. What we thought was a consumer business, actually turned out to just be a free business.

      One day we got a call from a major oil-and-gas company that had been using us, and we knew they were using us globally on oil rigs of their headquarters and other distributed offices. And then, finally, we got an email: “Look, we need to have a support contract, as a matter of our corporate hygiene. So, figure it out, and give us a quote. We need to have a way to call you if there’s a problem.”

      And so we went and got one of these virtual phone numbers on the internet that would route to an engineer’s phone number; and if that person didn’t pick up, it would route to the next engineer’s phone number; and if that person didn’t pick up, it would route to my phone.

      Sonal: Oh my, so you were like the support desk?

      David: Yeah, it was like a tiered call system. It went to three people, and we sent them a quote for one-hundred grand, and they signed it immediately and returned it. And now it went from us making $2 dollars a year in advertising — which we hated — to paying us $100,000 for something we’re already doing, and we get to turn off the ads.

      You don’t need to be a rocket scientist to figure out, wait a minute, maybe there’s something here. We had two or three other companies that had asked previously for something like that, so we went and sent them quotes, and they all signed them and returned them.

      Sonal: Why do you think you didn’t know that this would be the business model up front? Why did you have to pivot? Honestly, not to sound judgmental at all, but it seems obvious to me when you say it in hindsight <David: totally> — so I’m confused why you didn’t see that coming up.

      David: I think we were sort of enamored with this idea of keeping the whole internet safer, and that meant going after individuals. <Sonal: Idealistic> We had partnerships with Netgear and D-Link, and these people that sold consumer routers, and so we ignored the opportunity that’s right in front of our face.

      But as soon as you realize you’re not going to be able to raise money, and you actually have to build a business, you start to open your eyes a little bit. We did that, and then I hired Michelle Law, who actually spent seven years at Greylock, to run BD for us. Ultimately, she became my COO — a wonderful person, and a good friend — she had seen enterprise companies many times, and so she realized as we wanted to go enterprise that a bunch of the team had to change.

      First of all, half the team just didn’t care about building an enterprise software company, so they just quit. Then, the other half of the team just could not internalize that we can’t just change the UI overnight. Because it turns out some of our big customers had their own manuals that they had built with screenshots of our product. We got a nasty email once from this major oil-and-gas company that said, “We have all this training material and screenshots and videos we made, and you just totally changed your whole dashboard. You can’t do that.”

      You just have to learn how to manage those things, and then you do feature flags, which are things that are common today, but in 2009, feature flagging things and…

      Sonal: What’s feature flagging?

      David: Feature flagging means some subset or cohort of customers gets the access to a new feature, our new look and feel. A lot of people use it for A/B testing to see if something works, but you can also use it just to keep certain customers on certain packages, or on older features or an older look and feel. You still have one code base, but people have slightly different experiences.

      We were starting to do those things. We started implementing feature flags and things of that nature, but it meant that over the course of about 12-18 months, of the 30 people before the pivot, I think only 3 were left at the end.

      The role of CEO

      Sonal: When did you go from CEO to CTO?

      David: So right before all that happened, for most of 2008. The only good thing that ever came out of the total global recession and economic collapse was that my early investor needed cash, and so we found two investors (and that’s actually when I first met Mark and Ben) to come in and buy out my early investor. Those investors came in, started to rebuild the company.

      Sonal: But the biggest thing that’s fascinating to me is you came back as the CEO. So, what changed that you didn’t make this — I mean, ‘cause you’re the same person; you didn’t change overnight. <David: Right> Like, how did you… pull that off?

      David: Coming back as CEO the second time, after spending almost a year as CTO, one of the things I saw when I wasn’t CEO was all these things that weren’t happening in the company that should have been happening. Of course, I blamed the current CEO, but the reality was, I actually was not doing them either when I was CEO.

      Sometimes you just have to have this outside-the-glass-box kind of view, and you’re like: Wait a minute, people don’t know what’s important. Wait a minute, we’re not making it clear what the priorities are. Wait a minute, we’re not firing these low-performing people. But I wasn’t doing any of those things either!

      That, to me, was very eye-opening, so when I came back as CEO, I was a much better listener. I think I had this belief the first time I was CEO that I’m expected to have all the answers. It’s just not possible. What is important is CEOs have to make decisions, and I think they have to be able to articulate their decisions, but they don’t have to have all the answers, they don’t have to be the smartest person.

      Sonal: That’s a really important point… There’s actually a big difference between an answer and a decision. That’s actually something to reflect on, because I think most people conflate those two things.

      CEOs don’t have to have all the answers, but they do have to make decisions, and be able to articulate their decisions

      David: Totally. In fact, it turns out, actually, the opposite is true about having the answers. I often tell CEOs — because even before I joined Andreessen Horowitz, people called me for advice — and that’s one of the things I really enjoy.

      Sonal: It’s why you’re a VC now.

      David: Yeah, it’s why I’m a VC. That’s one of the best parts about the job. I like being the first phone call for a CEO when they’ve had the tough moment, or they need help.

      One of the things I often tell CEOs is: When you think about the table of leaders around you, there’s actually room around the table for one person who has no idea what they’re doing. And that’s you, as CEO.

      If you have the right leadership team, they’re adding intelligence for you.

      I had gone from technical CEO, to product CEO, to sales CEO, but my fault as a sales CEO is that that I loved the dog-and-pony show: I loved the pitch, I hated the close.

      Sonal: Why is that? That’s fascinating.

      David: You know I thought… you could say that it was ego or ignorance or naïveté. I didn’t like asking for the purchase orders because, first of all, I always thought our pricing was low. So if the customer… customers often like to negotiate.

      Sonal: Oh, and you’re ready to fight, like, “Fuck you, I want you to pay more.”

      David: Yeah, because the customer goes, “Oh, $100,000? I think it should be $70,000.” I’m like, “Fuck you, it should be $140,000.” <Sonal laughs> Like, “I’m raising the price!”

      Sonal: You’re the wrong guy to bring at the close, basically.

      David: My sales will be like, “Yeah, so, you can’t negotiate with this customer because you’re going to just blow up this deal.” I’m in a much different place now, obviously, but at that time, and where I was as a CEO, I hated the negotiation. I got uncomfortable.

      Everyone, they taught me so much. Because there was only room around the table for me to really not know all the answers.

      I will often say that my CMO at OpenDNS, Jeff Samuels — I think of him not just as my CMO but as a mentor to me — he taught me so much. I would say that about my VP of Engineering, my Head of Sales, my VP of Sales. I could take the inputs and use all that to make a decision, and I felt very good about those decisions I made. I think CEOs find it a huge relief when you tell them, you’re allowed to not know. In fact, if you have the best people, you’re going to know the least!

      It is not uncommon for a CEO or a leader, a manager… this is good general life advice. You don’t wanna be the whiny person constantly, like, harping on something, but I would say is that you do sometimes need to present an idea more than once. <Sonal: interesting> My old head of finance, who ultimately became my best friend, used to always joke with me that he would just tell me everything he said twice because he knew the first time, I’d ignore him.

      Sonal: <laughs> I think I have the same problem.

      David: He would tell me some statistic about what’s happening with marketing spend, or with hiring, or sales spends, and I’d be like, “Oh, that’s not really a problem. Like, whatever. I don’t care. You’re just a crazy finance bean-counter.” But then, he’d come back a week later. He’d be like, “Hey, I have more data. I did further analysis. You ignored me, but I know, I’m right here.” And I’d be like, “Oh, you’re right… why didn’t you tell me this last week?”

      Sonal: <both laugh> So, in that case, the CEO can get answers from all their management team and then make a decision based on all the answers you’re hearing.

      David: That’s right. And I do think sometimes you do have to tell people more than once, and that’s just the function of how human beings operate.

      Learning to listen

      Sonal: Speaking of this — telling people more than once and learning to listen — that was your big shift between when you came back to be CEO, and you kind of learned your lesson, so to speak.

      I honestly feel like that’s kind of a trite thing people say all the time. Like, listen better! I hate the “design-thinking” mindset around “empathy for this scenario” and “this persona”; it’s just so — I can’t diagnose what’s off…

      David: Yeah, I think when you’re building a company, being empathetic really means understanding — it doesn’t mean accommodating, right? This happens all the time as a leader, where you may not resolve that thing, but you can still understand it and be empathetic. I can be like, “Yeah, that is terrible. I understand what you’re saying, and I am hearing you.”

      Sonal: Honestly, that’s half the battle in relationships. <David: That’s right> You don’t need an answer to 99% of this. You just need someone to say, “Fuck, I feel for you. That sucks.” And you’re already feeling, like, 80% better.

      When you’re building a company, being empathetic really means understanding — it doesn’t mean accommodating

      David: That’s right, that’s right. When I think about empathy, you really want to be a great listener.

      A friend of mine, Wendy MacNaughton, she does this whole New York Times thing every other week where she really goes deep into a topic. She’s written these books. I think she thinks of herself as an artist. I think of her as an artist and entrepreneur. One of the things that she taught me a year or two ago was — when she’s trying to teach people how to be a really, really solid listener — is that when someone’s talking to you be like, “Tell me more about that.”

      Sonal: That’s just the phrase: “Tell me more about that.”

      David: Five words.

      Sonal: So, just make that your first question.

      David: When you’re talking to a customer, “Oh, what’s going on?” “Oh we’re doing annual planning?” It’s like, “Tell me more about that. How are you thinking about that? What is happening? What’s the frustrating part about annual planning? Tell me more about that.”

      Sonal: So what’s interesting to me about that is, to me, this is the difference between a focus group and an ethnographer. Focus groups and surveys are asking questions for things you already know to ask. <David: That’s right> An ethnographer is embedded in an organization or a setting and is essentially just listening to learn and observe, and letting those patterns reveal things.

      David: The deeper insights come out <Sonal: exactly> when you go down the tell-me-more-about-that path. That’s when you get these flashes in your brain of, wait, now I really understand what’s happening. It’s not that annual planning sucks, it’s that you’re having budget issues that aren’t being resolved in the way that you need, or that maybe your tools you’re using to do annual planning, or the way you communicate and collaborate with your team, or the way you work cross-functionally is not working.

      Sonal: Totally, totally. I consider myself an “ethnographer-esque editor”. <David: Totally> Because I want the context to know what I’m not hearing, to really understand.

      So, it’s interesting because, on the ethnographer side — I don’t think people know this about you, but you started off your career — or, academic career, because you actually started working when you were, like, what?

      David: Eighth grade, what is that, like, 14 or 15?

      Sonal: That’s when you got your first W-2, right?

      David: Yeah, I’ve had a 1099 or a W-2 every year since eighth grade. I worked at a mom and pop ISP in San Diego and learned all about routing and networking. I went to Washington University in St. Louis. I applied to the School of Arts and Sciences. When I went there to interview, they actually then had me interview with somebody in the School of Engineering in the Computer Science department. By the end of first semester of freshman year, I had switched back to the School of Arts and Sciences.

      And the reason I switched is I took a class “Introduction to Human Evolution”, and I just found it so fascinating. I’ve always learned in my life I do best in the things I really enjoy working on. I have trouble doing things I don’t want to do.

      Sonal: Me, too. I’m the exact same way.

      David: Yeah, it sounds obvious, but some people can actually just will their way through the other hard stuff.

      Sonal: No, I can’t. I can’t. I have no energy — I have zero. Talk about return on energy, I have no energy to even DO the thing.

      David: Yeah, I’m just like, I’ll be okay if I just don’t do this.

      Sonal: Yeah, I feel the exact same way.

      David: I had trouble in school with things that I really didn’t enjoy, so I learned how to optimize for the things that I like doing. And anthropology, like I just… every book I read I thought was so interesting. I learned about how women enforced power hierarchy in South America in a way that we don’t have elsewhere in the world. I learned about what happens in Southeast Asia around farming. I learned about the Green Revolution in Africa.

      And then I find that in my life, I actually think about these things all the time.

      Sonal: That was actually my next question, do you think it actually is useful in your career as a technologist?

      David: Oh, absolutely. Absolutely. I think about demographic transitions; when I read about what’s happening in Japan, it makes me think about how’s that going to change my investing thesis.

      I think it comes up constantly. It comes up both tactically as you think about yourself in leadership and organizational dynamics. It gives you an appreciation that there’s many perspectives in the world. In fact, it gives you an appreciation that more perspectives are better, and you want more.

      Sonal: So, that is a perfect way to close this episode. So, David Ulevitch — he’s made a journey from anthropologist, to technical CEO, to product CEO, to sales CEO, to go-to-market CEO — and now, investor. Welcome to the a16z Podcast.

      David: Thanks. Glad to be here.

      • David Ulevitch is a general partner at a16z where he invests in enterprise and SaaS companies. Prior to joining the firm, he was the founder and CEO of OpenDNS (acquired by Cisco).

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Innovating in Bets

      Annie Duke, Marc Andreessen, and Sonal Chokshi

      Every organization, whether small or big, early or late stage — and every individual, whether for themselves or others — makes countless decisions every day, under conditions of uncertainty. The question is, are we allowing that uncertainty to bubble to the surface, and if so, how much and when? Where does consensus, transparency, forecasting, backcasting, pre-mortems, and heck, even regret, usefully come in?

      Going beyond the typical discussion of focusing on process vs. outcomes and probabilistic thinking, this episode of the a16z Podcast features Thinking in Bets author Annie Duke — one of the top poker players in the world (and World Series of Poker champ), former psychology PhD, and founder of national decision education movement How I Decide — in conversation with Marc Andreessen and Sonal Chokshi. The episode covers everything from the role of narrative — hagiography or takedown? — to fighting (or embracing) evolution. How do we go from the bottom of the summit to the top of the summit to the entire landscape… and up, down, and opposite?

      The first step to understanding what really slows innovation down is understanding good decision-making — because we have conflicting interests, and are sometimes even competing against future versions of ourselves (or of our organizations). And there’s a set of possible futures that result from not making a decision as well. So why feel both pessimistic AND optimistic about all this??

      Show Notes

      • Using a football thought experiment to distinguish skill and luck [0:58]
      • Balancing outcomes and process [9:49]
      • Asking the right questions, especially with a negative outcome [11:17]
      • Discussion of timing in forecasting [15:23], and other practical implications [16:59]
      • Why not making a decision is also a decision [23:40], and how to evaluate the options you didn’t take [30:15]
      • Discussion of how widely this type of decision-making will be adopted by the public [34:10]
      • How to communicate probabilistically [37:24] and how to build uncertainty into an organization [40:21]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and today Mark and I are doing another one of our book author episodes. We’re interviewing Annie Duke, who’s a professional poker player and World Series champ, and is the author of “Thinking in Bets,” which is just out in paperback today. The subtitle of the book is, “Making Smarter Decisions When You Don’t Have All the Facts,” which actually applies to startups and companies of all sizes and ages, quite frankly. I mean, basically, any business or new product line operating under conditions of great uncertainty — which I would argue is my definition of a startup and innovation. So that will be the frame for this episode. 

      Annie’s also working on her next book right now, and founded howidecide.org, which brings together various stakeholders to create a national education movement around decision education, empowering students to also be better decision makers. So, anyway, Mark and I interview her about all sorts of things in and beyond her book, going from investing, to business, to life. But Annie begins with a thought experiment, even though neither of us really know that much about football.

      Skill vs. luck

      Annie: So what I’d love to do is, kind of, throw a thought experiment at you guys so that we can have a discussion about this. So I know you guys don’t know a lot about football, but this one’s pretty easy. You’re gonna be able to feel this one. I want you to do this thought experiment. Pete Carroll calls for Marshawn Lynch to actually run the ball.

      Sonal: So we’re betting on someone who we know is really good?

      Annie: Well, they’re all really good, but we’re betting on the play that everybody’s expecting.

      Mark: Yeah, the default.

      Annie: This is the default.

      Mark: The assumed rational thing to do, right?

      Annie: This is the assumed rational thing to do, right. So he has Russell Wilson hand it off to Marshawn Lynch. Marshawn Lynch goes to barrel through the line. He fails. Now they call the timeout — so now they stop the clock. They get another play now, and they hand the ball off to Marshawn Lynch — what everybody expects. Marshawn Lynch, again, attempts to get through that line and he fails. End of game, Patriots win. 

      My question to you is, are the headlines the next day, “The Worst Call in Super Bowl History”? Is Cris Collinsworth saying, “I can’t believe the call, I can’t believe the call.” Or is he saying something more like, “That’s why the Patriots are so good. Their line is so great. That’s the Patriots’ line that we’ve come to see this whole season. This will seal Belichick’s place in history.” It would’ve all been about the Patriots.

      So let’s, sort of, divide things into, like — we can either say the outcomes are due to skill or luck — and luck in this particular case is gonna be anything that has nothing to do with Pete Carroll. And we can agree that the Patriots’ line doesn’t have anything to do with Pete Carroll — Belichick doesn’t have anything to do with Pete Carroll — Tom Brady doesn’t have anything to do with Pete Carroll — as they’re sealing their fifth Super Bowl victory. 

      So what we can see is there’s two different routes to failure here. One route to failure, you get resulting. And basically what resulting is, is that retrospectively, once you have the outcome of a decision — once there’s a result — it’s really, really hard to work backwards from that single outcome to try to figure out what the decision quality is. This is just very hard for us to do. They say, “Oh my gosh, the outcome was so bad. This is clearly — I’m gonna put that right into the skill bucket. This is because of Pete Carroll’s own doing.” But in the other case, they’re like, “Oh, you know, there’s uncertainty. What could you do?” Weird, right?

      Sonal: Yeah.

      Annie: Okay, so you can kind of take that and you can say, “Aha, now we can, sort of, understand some things.” Like, for example, people have complained for a very long time that in the NFL they have been very, very slow to adopt what the analytics say that you should be adopting, right? And even though now we’ve got some movement on fourth-down calls, and when are you going for two-point conversions, and things like that, they’re still nowhere close to where they’re supposed to be, and why is that?

      Mark: So they don’t make the plays corresponding to the statistical probabilities?

      Annie: No. In fact, the analytics show that if you’re on your own one-yard line, and it’s fourth down, you should go for it no matter what. The reason for that is if you kick it, you’re only gonna be able to kick to midfield. So the other team is basically almost guaranteed three points anyway, so you’re supposed to just try to get the yards. Like, when have you ever seen a team on their own one-yard line on fourth down be like, “Yeah, let’s go for it.” That does not happen.

      Okay, so we know that they’ve been super slow to do what the analytics say is correct, and so you sit here and you go, “Well, why is that?” And that thought experiment really tells you why, because we’re all human beings. We all understand that there are certain times when we don’t allow uncertainty to bubble up to the surface — is the explanation — and there are certain times when we do. And it seems to be that we do, when we have this, kind of, consensus around the decision, there’s other ways we get there. And so, okay, if I’m a human decision-maker, I’m gonna choose the path where I don’t get yelled at.

      Sonal: Yeah, exactly.

      Annie: So, basically, we can, kind of, walk back, and we can say, “Are we allowing the uncertainty to bubble to the surface?” and this is gonna be the first step to, kind of, understanding what really slows innovation down — what really slows adoption of what we might know is good decision making, because we have conflicting interests, right? Making the best decision for the long run, or making the best decision to keep us out of a room where we’re getting judged.

      Mark: Yelled at, or possibly fired. So let me propose the framework that I use to think about this and see if you agree with it. So it’d be a two-by-two grid, and it’s consensus versus non-consensus, and it’s right versus wrong. And the way we think about it, at least in our business, is basically — consensus right is fine. Non-consensus right is fine. In fact, generally, you get called a genius. Consensus wrong is fine, because, you know, it’s just the same mistake everybody else made.

      Sonal: You all agreed, right, it was wrong.

      Mark: Non-consensus wrong is really bad.

      Annie: Horrible.

      Mark: It’s radioactively bad. And then as a consequence of that, and maybe this gets to the innovation stuff that you’ll be talking about — but as a consequence of that, there are only two scripts for talking about people operating in the non-consensus directions. One script is, they’re a genius because it went right — and the other is they’re a complete moron because it went wrong. Does that map?

      Annie: That’s exactly it. That’s exactly right. And I think that the problem here is that, what does right and wrong mean? In your two-by-two, wrong and right is really just, did it turn out well or not?

      Mark: Yeah, outcomes.

      Sonal: Not the process.

      Annie: And this is where we really get into this problem, because now what people are doing is they’re trying to swat the outcomes away. And they understand, just as you said, that on that consensus wrong, you will have a cloak of invisibility over you — like, you don’t have to deal with it. <Right.> So let’s think about other things besides consensus. So, consensus is one way to do that, especially when you have complicated cost-benefit analyses going into it. I don’t think that people, when they’re getting in a car, are actually doing any, kind of, calculation about what the cost-benefit analysis is to their own productivity, versus the danger of something very bad happening to them. Like, what is this society? Someone’s done this calculation, we’ve all, kind of, done this together — and so, therefore, getting in a car is totally fine. I’m gonna do that.

      Mark: And nobody second-guesses anybody. If somebody dies in a car crash you don’t say, “Wow, what a moron for getting in a car.”

      Annie: No. Another way that we can get there is through transparency. So if the decision is pretty transparent, another way to get there is status quo. So a good status quo example that I like to give, because everybody can understand it is — you have to get to a plane, and you’re with your significant other in the car, and you go the usual route.

      Sonal: This is a common fight for every couple.

      Annie: Yeah, so you go your usual route. Literally, this is the route that you’ve always gone and there is some sort of accident, there’s bad traffic, you miss the plane — and you’re mostly probably comforting each other in the car. It’s like, “What could we do?” You know, eh. But then you get in the car and you announce to your significant other, “I’ve got a great shortcut, so let’s take this shortcut to the airport.” And there’s the same accident, whatever — horrible traffic, you miss the flight. That’s like that status quo versus non-status quo decision.

      Sonal: Right, you’re going against what’s familiar and comfortable.

      Annie: Exactly. If we go back to the car example, when you look at what the reaction is to a pedestrian dying because of an autonomous vehicle, versus because of a human, we’re very, very harsh with the algorithms. For example, if you get in a car accident and you happen to hit a pedestrian, I can say something like, “Well, you know, Mark didn’t intend to do that.” Because I think that I understand — your mind is not such a black box to me. So I feel like I have some insight into what your decision might be, and so more allowing some of the uncertainty to bubble up there. But if this black box algorithm makes the decision, now all of a sudden I’m like, “Get these cars off the road.”

      Sonal: Never mind that the human mind is a black box itself ultimately, right?

      Annie: Of course, but we have some sort of illusion that I understand, sort of, what’s going on in there, just like I have an illusion that I understand what’s going on in my own brain. And you can actually see this in some of the language around crashes on Wall Street, too. When you have a crash that comes from human beings selling, people say things like, “The market went down today.” When it’s algorithms, they say, “It’s a flash crash.” So now they’re, sort of, pointing out, like — this is clearly in the skilled category. It’s the algorithm’s fault. We should really have a discussion about algorithmic trading and whether this should be allowed, when obviously the mechanism for the market going down is the same either way.

      So now if we understand that, so exactly your matrix. Now we can say, “Well, okay, human beings understand what’s gonna get them in the room.” And pretty much anybody who’s, you know, living and breathing in the top levels of business at this point is gonna tell you, “Process, process, process. I don’t care about your outcomes — process, process, process.” But then the only time they ever have, like, an all-hands-on-deck meeting is when something goes wrong. Let’s say that you’re in a real estate investing group, and so you invest in a particular property based on your model, and the appraisal comes in 10% lower than what you expected. Like, everybody’s in a room, right? You’re all having a discussion. You’re all examining the model, you’re trying to figure out. But what happens when the appraisal comes in 10% higher than expected? Is everyone in the room going, “What happened here?”

      Outcomes vs. process

      Mark: Now there is the obvious reality, which is, like, we don’t get paid in process, we get paid in outcomes. Poker players, you don’t get paid in process, you get paid in outcome, and so there is a…

      Sonal: Incentive alignment.

      Mark: It’s not completely emotional. It’s also an actual — there’s a real component to it.

      Annie: Yeah, so two things. One is, you have to make it very clear to the people who work for you that you understand that outcomes will come from good process. That’s number one. And then number two, what you have to do is try to align the fact that, as human beings, we tend to be outcome driven — to what you want, in terms of getting an individual’s risk to align with the enterprise risk. Because otherwise you’re gonna get this CYA behavior. And the other thing is that we wanna understand if we have the right assessment of risk. So one of the big problems with the appraisal coming in 10% too high, there, could be that your model is correct. It could be that you could have just a tail result, but it certainly is a trigger for you to go look and say, “Was there risk in this decision that we didn’t know was there?” And it’s really important for deploying resources.

      Sonal: I have a question about translating this to, say, non-investing context. So in the example of Mark’s matrix, even if it’s a non-consensus wrong — you are staking money that you are responsible for. In most companies, people do not have that kind of skin in the game. <Right.> So how do you drive accountability in a process-driven environment — that the results actually do matter? You want people to be accountable, yet not overly focused on the outcome. Like, how do you calibrate that?

      Annie: So let’s think about, how can we create balance across three dimensions that makes it so that the outcome you care about is the quality of the forecast? So first of all, obviously this demands that you have people making forecasts. You have to state in advance, “Here’s what I think. This is my model of the world. Here are where all the places are gonna fall. So this is what I think.” So now you stated that, and whether the outcome is “good or bad” is — how close are you to whatever that forecast is?

      So, now it’s not just like, oh, you won to it, or you lost to it. It’s — was your forecast good? So that’s piece number one, is make sure that you’re trying to be as equal across quality as you can, and focus more on forecast quality as opposed to traditionally what we would think of as outcome quality. So now the second piece is directional. So, when we have a bad outcome and everybody gets in the room, when was the last time that someone suggested, “Well, you know, we really should’ve lost more here?” Like, literally nobody’s saying that, but sometimes that’s true. Sometimes if you examine it, you’ll find out that you didn’t have a big enough position. It turned out, okay, well, maybe we should’ve actually lost more. So you wanna ask both up, down, and orthogonal. So, could we have lost less? Should we have lost more? And then the question of, should we have been in this position at all?

      Mark: So in venture capital, after a company works and exits — let’s say it sells for a lot of money, you do often say, “God, I wish we had invested more money.” You never, ever, ever, ever — I have never heard anybody say on a loss, “We should’ve invested more money.”

      Annie: See, wouldn’t it be great if someone said that? Wouldn’t you love for someone to come up and say that to you? That would make you so happy.

      Sonal: I actually still don’t get…

      Mark: And what would be the logic of why they should say that?

      Sonal: I still don’t get the point. Exactly. Why does that matter? I don’t really understand that.

      Annie: Can I just, like — simple, in a poker example?

      Sonal: Yeah.

      Annie: So let’s say that I get involved in a hand with you, and I have some idea about how you play. And I have decided that you are somebody that, if I bet X, you will continue to play with me. Let’s say this is a spot where I know that I have the best hand, but if I bet X plus C that you will fold. So if I go above X, I’m not gonna be able to keep you coming along with me, but if I bet X or below, then you will — so I bet X. You call, but you call really fast, in a way that makes me realize, “Oh, I could’ve actually bet X plus C.” You hit a very lucky card on the end, and I happen to lose the pot. I should’ve maximized at the point that I was the mathematical favorite.

      Mark: Because your model of me was wrong, which is a learning independent of the win or the loss.

      Annie: Exactly. So you need to be exploring those questions in a real honest way.

      Mark: Right, because it has to do with how you size future bets.

      Sonal: This is exactly like a company betting on a product line.

      Annie: Correct.

      Sonal: And then picking what the next product line is gonna be, and then not having had the information that would then drive a better decision-making process around that.

      Annie: Right. So thinking about the learning loss that’s happening because we’re not exploring that — the negative direction — and now you should do this on wins as well. So if you do ever discuss a win, you always think, like, “How could I press? How could I have won more? How could I have made this even better? How could I do this again in the future? Should we have won less?”

      Mark: We oversized the bet and then got bailed out by a fluke.

      Annie: We should have actually had less in it, and sometimes not at all, because sometimes the reasons that we invested turned out to be orthogonal to the reasons that it actually ended up playing out in the way that it was. And so, had we had that information, we actually wouldn’t have bet on this at all because it was completely orthogonal. We totally had this wrong. It just turned out that we ended up winning. And that can happen. I mean, obviously that happens in poker all the time, but what does that communicate to the people on your team? 

      Good, bad, I don’t care. I care about our model. I wanna know that we’re modeling the world well, and that we’re thinking about, “How do we incorporate the things that we learn?” Because we can generally think about stuff in two — stuff we know, and stuff we don’t know. There’s stuff we don’t know we know, obviously — so we don’t worry about that, because we don’t know we don’t know it. But then there’s stuff we could know, and stuff we can’t know. It’s things like the size of the universe, or the thoughts of others, <Exactly.> or what the outcome will actually be. We don’t know that.

      Sonal: I have a question about this, though. What is the timeframe for that forecast? So let’s say you have a model of the world — a model of a technology, how it’s gonna adopt, how it’s gonna play out. In some cases, there are companies that can take, like, years to get traction. You wanna get your customers very early to figure that out, right? So you can get that data. But how much time do you give? How do you size that timeframe for the forecast, so you’re not constantly updating with every customer data point, and so you’re also giving it enough time for your model, your plan, your forecast to play out?

      Annie: You have to think about —very clearly in advance, “What’s my time horizon? How long do I need for this to play out?” But also, don’t just do this for the big decisions — because there’s things that you can forecast for tomorrow as well, so that you end up bringing it into just the way that people think. And then once you’ve decided, “Okay, this is the time horizon on my forecast,” then you would wanna be thinking about, “What are forecasts we make for a year, two years, five years for this specific decision to play out?” And then just make sure that you talk in advance — at what point you’ll revisit the forecast. So you wanna think in advance, “What are the things that would have to be true for me to be willing to come in and actually revisit this forecast?” Because otherwise, you can start, as you just said, like — it can turn into — super bad.

      Sonal: You’re like a leaf in the wind. Right, exactly, because then you’re, like, one bad customer and you suddenly over-rotate on that — when in fact, it could’ve been not even a thing.

      Annie: Right, so if you include that in your forecast — here are the circumstances under which we would come in and check on our model — then you’ve already gotten that in advance. So that’s actually creating constraints around the re-activity, which is helpful.

      Barriers to logical decision-making

      Mark: Two questions on practical implementation of the theory. So what I’m finding is, more and more people understand the logic of what you’re describing, because people are getting exposed to these ideas and, kind of — expanding in importance. And so more and more people intellectually understand this stuff, but there’s two, kind of — I don’t know, so call it emotion-driven warps, or something — that people just really have a hard time with. So one is that you understand this could be true investors, CEO, product-line manager in a company — you know, kind of, anybody in one of these domains — which is you can’t get the non-consensus results unless you’re willing to take the damage, right, the risk on the non-consensus wrong results.

      But people cannot cope with the non-consensus wrong outcome. They just emotionally cannot handle it — and they would like to think that they can, and they intellectually understand that they should be able to. But as you say, when they’re in the room it’s such a traumatizing experience that it’s the “touching the hot stove.” They will do anything in the future to avoid that. And so one interpretation would be, that’s just simply flat out human nature — and so, to some extent, the intellectual understanding here — it doesn’t actually matter that much, because there is an emotional override. And so that would be a pessimistic view on our ability as a species to learn these lessons, or do you have a more optimistic view of that?

      Annie: I’m gonna be both pessimistic and optimistic at the same time, so let me explain why.

      Sonal: Ooh, love it.

      Annie: Because I think that if you move this a little bit it’s a huge difference. You, sort of, have two tacks that you wanna take. One is, how much can you move the individual to, sort of, train this kind of thinking for them? And that means, naturally, they’re thinking in forecasts a little bit more — that when they do have those kinds of reactions, which naturally everybody will, they right the ship more quickly, so that they can learn the lessons more quickly, right? I mean, I actually just had this happen. I turned in a draft of my next book — the first part of my next book to my editor — and I just got the worst comments I’ve ever gotten back.

      Sonal: Good editor.

      Annie: And I had a really bad 24 hours, but after 24 hours, I was like, “You know what? She’s right.” Now, I still had a really bad 24 hours — and I’m the, like, “give me negative feedback” queen. Because I’m a human being. But I got to it fast. I, sort of, got through it pretty quickly after this. I mean, I — you know, on the phone with my agent saying, “I’m standing my ground, this is ridiculous.” And then he got a text the next day being, like, “No, she’s right.” And then I rewrote it, and you know what? It’s so much better for having been re-written, and now I can get to a place of gratitude for having the negative feedback. But I still had the really bad day, so it’s okay.

      Sonal: So, it doesn’t go away, right?

      Annie: Yeah, and it’s okay. We’re all human, we’re not robots. So number one is, like, how much are you getting the individuals to say, “Okay, I improved 2%, that’s so amazing for my decision making and my learning going forward?” And then the second through-line is, what are you doing to not make it worse? Because obviously for a long time people liked to talk about, “I’m results oriented.” That’s, like, the worst sentence that could come out of somebody’s mouth.

      Sonal: Why is that the worst? I’ve heard that a lot, what’s so bad about it?

      Annie: Because you’re letting people know that all you care about is, like, “Did you win or lose?” That’s fantastic — be results oriented all you want. You should pay by the piece. You will get much faster work. But the minute that you’re asking people to do intellectual work, results oriented is, like, the worst thing that you could say to somebody. So I think that we need to take responsibility, and the people in our orbit — we can make sure at minimum that we aren’t making it worse. And I think that that — so that’s pessimistic and optimistic. I don’t think anyone is making a full reversal here.

      Mark: So the second question then goes to the societal aspect of this. And so we’ll talk about the role of the storytellers — or as they’re sometimes known, the journalists.

      Annie: Yeah, and the editors.

      Sonal: I love it.

      Mark: And the editors, and the publishers. And so the very first reporter I ever met when I was a kid — Jared Sandberg at the Wall Street Journal — you know, the internet was first emerging. There were no stories in the press about the internet, and I used to say, “There’s all this internet stuff happening. Why am I not reading about any of it in any of these newspapers?” And he’s like, “Well, because the story of ‘something is happening’ is not an interesting story.” He said, “There are only two stories that sell newspapers.” He said, “One is, ‘Oh, the glory of it,’ and the other is, ‘Oh, the shame of it.’” And basically he said it’s conflict. So it’s either something wonderful has happened, or something horrible has happened, <Yeah.> those are the two stories. And then you think about business journalism as, kind of, our domain — and you kinda think about it, and it’s like, those are the only two profiles of a CEO or founder you’ll ever read.

      It’s just, like, what a super genius for doing something presumably non-consensus and right, or what a moron. Like, what a hopeless idiot for doing something non-consensus and wrong. And so, since I’ve become more aware of this, it’s gotten very hard for me to actually read any of the coverage of the people I know, because — it’s like the people who got non-consensus right, they’re being lavished with too much praise. <Ah.> And the people who got non-consensus wrong, they’re being damned for all kinds of reasons. The traits are actually the same in a lot of cases. And so, I guess, as a consequence — if you read the coverage, it really reinforces this bias of being results-oriented. And it’s like, it’s not our fault that people don’t wanna read a story that says, “Well, you tried something and it didn’t work this time,” right?

      Annie: Yes, exactly. But it was mathematically pretty good. If we go back to Pete Carroll, this is a pretty great case. And if we think about options theory, just quickly — the paths preserve the option for two run plays. So if you wanna get three tries at the end zone instead of two, strictly for clock management reasons, you pass first.

      Mark: Right, and that’s not gonna kick off ESPN “SportsCenter” that night. And so optimistic or pessimistic that the narrative — the public narrative on these topics will ever move? 

      Annie: I’m super, super pessimistic on the societal level, but I’m optimistic on — if we’re educating people better, that we can equip them better for this. So I’m really focused on, how do we make sure that we’re equipping people to be able to parse those narratives in a way that’s more rational? And particularly, you know — now there’s so much information, and it’s all about the framing, and the storytelling — and it’s particularly driven by, what’s the interaction of your own point of view? We could think about it as [a] partisan point of view, for example, versus the point of view of the communicator of the information, and how is that interacting with each other. You know, in terms of, how critically are you viewing the information, for example? I think this is another really big piece of the pie, and somewhat actually related to the question about journalism, which is that third dimension of the space.

      So we talked about two-dimension, which is, sort of, outcome quality, and how are you allowing that you’re exploring both downside and upside outcomes in a way that’s really looking at forecast? How are you thinking directionally, so that you’re more directionally neutral? But then the other piece of the puzzle is, how are you treating omissions versus commissions? 

      So one of the things that we know with this issue of resulting is, here’s a really great way to make sure that nobody ever results on you — don’t do anything, okay? So if I just don’t ever make a decision, I’m never gonna be in that room with everybody yelling at me for the stupid decision I made, because I had a bad outcome. But we know that not making a decision is making a decision, we just don’t think about it that way. And it doesn’t have to just be bad investing. You can have a shadow book of your own personal decisions.

      Sonal: Personal life, I agree.

      Not making a decision is a decision

      Annie: So, you know, it’s really interesting — I remember I was giving somebody advice, who — I think he was, like, 23. And so, obviously, newly out of college, had been in this position for a year, and was really, really unhappy in the position. And he was asking me, like, “I don’t know what to do. I don’t know if I should change jobs.” And I said, “Well…” So I did all the tricks, you know, time traveling — and so I was like, “Okay, imagine it’s a year from now. Do you think you’re gonna be happy in this job?” “No.” “Okay, well, maybe you should choose this other — go and try to find another position.” And this is what he said to me — and this, I think, shows you how much people don’t realize that the thing that you’re already doing, the status quo thing — choosing to stay in that really is a decision.

      So he said to me, “But if I go and find another position, and then I have to spend another year, which I just spent, trying to learn the ins and outs of the company, and it turns out that I’m not happy there, I’ll have wasted my time.” And I said to him, “Okay, well, let’s think about this, though. The job you’re in, which is a choice to stay in, you’ve now told me it’s 100% in a year that you will be sad. Then if you go to the new job, yes, of course it’s more volatile — but at least you’ve opened the range of outcomes up.” But he didn’t wanna do it because it doesn’t feel — like, staying where he was didn’t feel like somehow he was choosing it, so that he felt like if he went to the other place <Yes.> and ended up sad that somehow that would be his fault and a bad decision.

      Sonal: That’s so, so profound. In my case — this might be getting a little too personal, but in my case it was a decision I didn’t know I had made, to not have kids. And it’s still an option, but it’s probably not gonna happen. And my therapist, kind of, told me that my not deciding was a choice — and I was so blown away by that that it, then, allowed me to then examine what was going on there in that framework, in order to not do that for other arenas in my life where I might actually want something. Or maybe I don’t, but at least it’s a choice, that there’s intentionality behind it.

      Annie: Well, I appreciate you sharing. I mean, I really wanna thank you for that, because I think that people, first of all, should be sharing this kind of stuff so that people feel like they can talk about these kinds of things, number one.

      Sonal: I agree.

      Annie: And number two, in my book, I’ve got all these examples in there of, like — how are you making choices about raising your kids when it feels so consequential?

      Sonal: When you’re doing decisions for other people?

      Annie: Right, and you’re trying to decide, like, “Should I have kids, or shouldn’t I have kids?”

      Sonal: Or this school, or that school?

      Annie: Or, “Who am I supposed to marry, or where am I supposed to live?” And the thing that I try to get across is, you know — we can talk about investing, like, I’m putting money into some kind of financial instrument, but we all have resources that we’re investing. It’s our time.

      Sonal: That’s right. Your time, your energy, your heart. It could be whatever, your friendships, your relationships.

      Annie: Right, so you’re deploying resources.

      Sonal: Yes, I love that.

      Annie: And for the kind of decision that you’re talking about, it’s like — if you choose to have children you’re choosing to deploy certain resources with some expected return. Some of it good, some of it bad. And if you’re choosing not to have children, that’s a different deployment of your resources toward other things.

      Sonal: And you need to know that there are limits. Everything isn’t a zero-sum game, <No.> but approaching the world, and the fact that evolution has approached the world as a zero-sum game — and our toolkit makes it a zero-sum game — means that we need to still view everything as a zero-sum game when it comes to those tradeoffs and resources. Because you are losing something every time, even in a non-zero game.

      Annie: Right. So I don’t feel like the world is a zero-sum game in terms of, like, <Collaborate, coordinate.> most of the activities that you and I would engage around, we can both win, too. But it’s a zero-sum game, to go back to your therapist. It’s a zero-sum game between you and the other versions of yourself that you don’t choose.

      Sonal: Exactly. Or an organization, and the other versions of itself it doesn’t choose.

      Annie: Exactly. So there’s a set of possible futures that result from not making a decision as well. So on an individual decision, let’s put things into three categories: clear misses, near misses, and hits. There’s some that would just be a clear miss — throw them out — and there’s some that I’m gonna, sort of, really agonize over and I’m gonna, you know, think about it, and I’m gonna do a lot of analysis on it. So the ones which become a yes go into the hit category, and the other one is a near miss. I came close. What happens with those near misses is they just go away. 

      So what I realized is that on any given decision — let’s take an investment decision. If I went to you, or you came to me, and said, “Well, tell me what’s happening with the companies that you have under consideration.” On a single decision, when I explain to you why I didn’t invest in a company, it’s gonna sound incredibly reasonable to you.

      So you’ll only be able to see in the aggregate, if you look across many of those decisions, that I tend to be having this bias toward missing — towards saying, “You know what? We’re not gonna do it,” so that I don’t wanna stick my neck out. Now this, for you, is incredibly hard to spot because you do have to see it in the aggregate. Because I’m gonna be able to tell you a very good story on any individual decision. So the way to combat that — and again, get people to think about, “What we really care [about] around here is forecast, not really outcomes” — is actually to keep a shadow book. The anti-portfolio should contain basically all of your near misses, but then you have to take a sample of the clear misses as well — which nobody ever looks at. Because the near misses tend to be a little in your periphery if they happen to be big hits.

      Mark: So the good news, bad news. So the good news is we have actually done this, and so we call it the shadow portfolio. <Awesome.> And the way that we do it is, we make the investment. We take an equivalent. We take the other equivalent deal of that vintage, of that size — that we almost did but didn’t do — and we put that in the shadow portfolio. And we’re trying to do, kind of, apples-to-apples comparison. In finance theory terms, the shadow portfolio may well outperform the real portfolio, and in finance terms that’s because the shadow portfolio may be higher variance. Higher volatility, higher risk, and therefore, higher return.

      Annie: Correct.

      Mark: Because, right, the fear is the ones that are hitting are the ones that are less spiky, they’re less volatile, they’re less risky.

      Annie: Right. So what’s wonderful about that, when you decide not to invest in a company, you actually model out why. That’s in there.

      Mark: It’s often, by the way, a single flaw that we’ve identified. It’s just like, oh, we would do it except for X, <Right.> where X looks like something that’s potentially existentially bad.

      Annie: Right, and then that’s just written in there, and so you know that. And then, just make sure those ones that people are just rejecting out of hand, a sample, just a sample.

      Mark: Okay, so that’s my question. So we never do that. Let me ask you how to do that, though. So that’s what we don’t do, and as you’re describing, I’m like, “Of course we should be doing that.” I’m trying to think of how we would do that, because the problem is, we reject 99 for every 1 we do.

      Annie: Yeah, so you just — literally it’s a sample. You just take a random sample of them.

      Mark: A random sample? Okay.

      Annie: I mean, as long as it’s just, sort of, being kept in view a little bit. Because what that does is it basically just asks as — pushing against your model. You’re just, sort of, getting people to have the right kind of discussion. So all of that communicates to the people around you, like, “I care about your model.”

      Evaluating options you didn’t take

      Mark: So let me ask you a different question because you talk about these, sort of, groups of decisions, or portfolios of decisions. So the other question is — so early on in the firm, I happened to have this discussion with a friend of mine, and he basically looked at me and was like, “You’re thinking about this all wrong. You are thinking about this as a decision. You’re thinking about, ‘Invest or not?'” He said, “That’s totally the wrong way to think about this. [The way] you should be thinking about this is, is — is this 1 of the 20 investments of this kind, or of this class size, that you’re gonna put in your portfolio?” When you’re evaluating an opportunity, you are, kind of, definitionally talking about that opportunity. But it’s very hard to abstract that question from the broader concept of a portfolio or a basket.

      Annie: Yeah, what I would suggest there is actually just doing some time traveling. That as people are really down in the weeds, to say, “Let’s imagine it’s a year from now, and what does the portfolio look like of these investments of this kind?” So I’m a big proponent of time traveling — of just making sure that you’re always asking that question, “What does this look like in a year? What does this look like in five years? Are we happy? Are we sad? If we imagine that we have this, what percentage of this do we think will have failed? We understand that any one of these individual ones could have failed, so let’s remember that.” And I think that that really allows you to, sort of, get out of what feels like the biggest decision on earth, because that’s the decision you have to be making, and be able to see it in the context of, kind of, all of what’s going on.

      Sonal: That’s fantastic. One of the most powerful things my therapist gave me — and it was such a simple construct. It was, sort of, like, doing certain things today is like stealing from my future self. It blew my mind.

      Annie: It’s so beautiful.

      Sonal: It’s so beautiful. And it seems so, like, you know. Hokey — like, personal, self-helpy — but actually I had never thought of [it]. Because we’re on a continuum. By making discrete individuals — like, Sonal in the past, Sonal today, Sonal, this woman in the future I haven’t met yet. Wow. Like, the idea of stealing from her was, like…

      Annie: That’s really a lovely way to put it.

      Sonal: Isn’t that so — she’s a fucking awesome therapist, for the record.

      Annie: Yeah, she is. I have an amazing therapist.

      Sonal: I like talking publicly about therapy because I like lifting the stigma on it.

      Annie: No, I’m very, very open about it.

      Sonal: Me too.

      Annie: Like, let’s not hide it. It’s totally fine.

      Sonal: No. There’s no fucking reason to hide it, I totally agree.

      Annie: Yeah. Some of the ways that we deal with this is actually prospectively, employing really good decision hygiene — which involves a couple of things. One is some of this good time traveling that we talked about, where you’re really imagining, “What is this gonna look like in the future,” so that that’s metabolized into the decision. Two is making sure that you have pushback once there’s consensus reached. Great, now let’s go disagree with each other. Then the next thing is, in terms of the consensus problem, is to make sure that you’re listening [to] as much input, not in a room with other people. So when somebody has a deal they wanna bring to everybody, that goes to the people individually. They have to, sort of, write their thoughts about it individually, and then it comes into the room after that.

      Mark: As opposed to the pile-on effect that tends to happen?

      Annie: As opposed to the pile-on effect, and that reduces the sort of effects of consensus anyway. So now this is how you then come up with basically what your forecast of the future is, that then is absolutely memorialized. Because that memorializing of it acts as the prophylactic. First of all, it gives you your forecast, which is what you’re trying to push against anyway. You’re trying to change the attitude to be that the forecast is the outcome that we care about. And it acts as a prophylactic for those emotional issues, right?

      Which is now it’s like, okay, well, we all talked about this, and we had our red team over here, and we had a good steel man going on, and we, kind of, really thought about why we were wrong. We questioned — if someone has the outside view, what would this really look like to them? By eliciting the information individually, we were less likely to be in the inside view anyway. We’ve done all of that good hygiene — and then that acts as a way to protect yourself against these kinds of issues in the first place. Again, you’re gonna have a bad 24 hours, I’m just saying. Like, for sure. But you can get out of it more quickly, more often, and get to a place where you can say, “Okay, moving onto the next decision. How do I improve this going forward?”

      Sonal: You make better and better decisions.

      Mark: Yeah, so building on that, but returning real quick to my optimism, pessimism question. If society is not going to move on these issues, but we can move as individuals — so one form of optimism would be, more of us move as individuals. The other form of optimism could be, there will just always be room in these probabilistic domains for the rare individual who’s actually able to think about this stuff correctly. There will always be an edge. There will always be certain people who are, like, much better at poker than everybody else.

      Annie: Oh, I think that’s for sure.

      Mark: Okay. Because most people just simply can’t or won’t get there. Like, a few people in every domain might be able to take the time and have the discipline and will power to, kind of, get all the way there, but most people can’t or won’t?

      Annie: I think that, in some ways, maybe that’s okay. Like, I mean, I sort of think about it from an evolutionary standpoint. That kind of thinking was selected for for a reason, right? It’s better for survival, likely better for happiness.

      Mark: You mean the conventional wisdom of “don’t touch the burning stove twice.”

      Annie: Yeah, or run away when you hear rustling in the leaves. Don’t sit around and say, “Well, it’s a probabilistic world. I have to figure out, how often is that a lion that’s gonna come eat me?”

      Mark: Most people shouldn’t be playing in the World Series of Poker.

      Annie: I have people come up to me all the time and be like, “Oh, you know, I play poker but it’s just a home game,” you know? And I’m like, “Why do you say ‘just a home game?’ There are different purposes to poker. You probably have a great time doing that and it brings you a tremendous amount of enjoyment, and you don’t have an interest in becoming a professional poker player. Just be proud of that, I think that that’s amazing.” Like, I play tennis. I’m not saying, “Oh, but, you know, I’m just playing in USTA 3.5.” I’m really happy with my tennis, I think it’s great.

      So I think we need to remember that people have different things that they love. And this kind of thinking, I think that — I would love it if we could spread it more — but of course there are gonna be some people who are going to be ending up in this category more than others, and that’s okay. Not everybody has to think like this. I think it’s all right. So one of the things I get asked all the time is, like, “Well, we can’t really do this because people expect us to be confident in our choices.” <Yes.> Don’t confuse confidence and certainty. So, I can express a lot of uncertainty and still convey confidence. Ready? I’m weighing these three options: A, B, and C. I’ve really done the analysis. Here’s the analysis, and this is what I think. I think that option A is gonna work out 60% of the time. Option B is gonna work out 25% of the time, and option C is gonna work out 15% of the time. So option A is the clear winner. Now I just expressed so much uncertainty in that sentence.

      Sonal: But also a lot of confidence.

      Annie: But also a lot of confidence. I’ve done my analysis, this is my forecast. And all that I ever ask people to do when they do that is make sure that they ask a question before they bank the decision, which is — is there some piece of information that I could find out that would reverse my decision, that would actually cause — not that would make it go from 60 to 57. I don’t care modulating so much, I care that you’re gonna actually change.

      Sonal: Right. And your point is that organizations can then bake that into their process.

      Annie: Correct.

      Sonal: And not just in the forecasting, but in arriving to that decision. So that then the next time they get to it, right or wrong, they make a better decision.

      Annie: Right. And if the answer is yes, go find it. Or sometimes the answer is yes, but the cost is too high. It could be time, it could be actual…

      Sonal: Opportunity costs, etc., right.

      Annie: Whatever, exactly. So then you just don’t, and then you would say, “Well, then you all recognize as a group, we knew that if we found this out it would change our decision. But we’ve agreed that the cost was too high and so we didn’t.” So then if it reveals itself afterwards, you’re not sad.

      Communicating probability to others

      Sonal: Yeah, right. Well, you’ve talked a lot about how people should use confidence intervals in communicating — which I love, because we’re both ex-Ph.D psychology people.

      Annie: Yes, exactly.

      Sonal: Neither finished. So I love that idea. One thing that I struggle with, though, is — again, in the organizational context. If you’re trying to translate this to a big group of people, not just one on one or small group decisions. How do you communicate a confidence interval, and all the variables in it, in an efficient, kind of, compressed way? Like, honestly, part of communication in organizations is emails, and quick decisions — and yes, you can have all the process behind the outcome, but how do you then convey that, even though the people were not part of that room, of that discussion?

      Annie: I think that there’s a simpler way to express uncertainty, which is using percentages. Now, obviously, sometimes you can only come up with a range. But for example, if I’m talking to my editor — and this is very quick in an email, I’ll say, “You’ll have the draft by Friday 83% of the time — by Monday, you’ll have it 97% of the time.” Those are inclusive, right?

      Sonal: It’s another way of doing a confidence interval, but without making it so wonky.

      Annie: Without making it so wonky. So I’m just letting her know — most of the time you’re gonna get it on Friday but I’m building, like, if my kid gets sick, or I have trouble with a particular section of the draft — or whatever it is — and I set the expectations for it that way.

      Sonal: That’s fantastic. I mean, we’ve been trying to do forecasting — even for, like, timelines for podcast editing in episodes. And I feel frustrated, because I have a set of frameworks — like, if there’s accents, if there’s more than two voices. If there’s a complex thing, room tone, interaction, feedback, sound effects. I know all the factors that can go into my model, but I don’t know how to put a confidence interval in our pipeline spreadsheet for all the content that’s coming out and predicting it.

      Annie: Yeah, so one way to do it is think about — what’s the range? What’s the earliest that I could get it? And you put a percentage on that. And then you think about the latest day they’re gonna get it, and you put a percentage on that.

      Sonal: I love that idea.

      Annie: And so now, what’s wonderful about that is that — it’s a few things. One is, I’ve set the expectations properly now, so that I’m not getting, you know, yelled at on Friday, like, “Where the hell is the draft?”

      Sonal: Exactly, which is half the battle, I’ve learned that.

      Annie: And a lot of what happens is that because we think that we have to give a certain answer, it ends up “boy who cried wolf,” right? So that if I’m telling her I’m gonna get it on Friday, and, you know, 25% of the time…

      Sonal: Honestly, against your own best judgment sometimes even.

      Annie: Right, 25% of the time I’m late, she just starts to not put much stock in what I’ve said anyway. So that’s number one. Number two is — what happens is that you really, kind of, infect other people with this in a good way, where you get them — it just moves them off of that black and white thinking.

      Sonal: I love that.

      Annie: So, like, one of the things that I love thinking about — and this is the difference between a deadline or, kind of, giving this range — is that I think that we ask ourselves, “Am I sure?” and other people, “Are you sure?” way too often. It’s a terrible question to ask somebody because the only answer is yes or no.

      Sonal: So what should we be asking?

      Annie: How sure are you?

      Uncertainty in an organization

      Sonal: How sure are you? I have a quick question for you on this, because earlier you mentioned uncertainty. How do you as an organization build that uncertainty in by default?

      Annie: So first of all, we obviously talked a little bit about time traveling and the usefulness of time traveling. So one thing that I like to think about is not [to] overvalue the decision that’s right at hand — the things that are right sitting in front of us, right? So you can kind of think about it, like, how are you gonna figure out the best path? What is it, as you think about what your goals are? And, obviously, the goal that you wanna reach is gonna, sort of, define for you what the best path is. 

      If you’re standing at the bottom of a mountain that you wanna summit — let’s call the summit your goal — all you can really see is the base of the mountain. So as you’re doing your planning, you’re really worried about, “How do I get the next little bit,” right? “How do I start?” But if you’re at the top of the mountain, having attained your goal, now you can look at the whole landscape. You get this beautiful view of the whole landscape, and now you can really see what the best path looks like. And so we wanna do this not just physically — like, standing up on a mountain, but we wanna figure out a cognitive way to get there, and that’s to do this really good time traveling. 

      And you do this through backcasting and premortem. And now let’s look backwards, instead of forwards, to try to figure out — this is now the headline. Let me think about why that happened. So you could think about this as a simple weight-loss goal. I wanna lose a certain amount of weight within the next six months. It’s the end of the six months, I’ve lost that weight. What happened? I went to the gym, I avoided bread, I didn’t eat any sweets. I made sure that, you know, whatever. So you now have this list. Then in pairing with that, you wanna do a premortem, which is — I didn’t get to the top of the mountain. I failed to lose the weight. I failed to do whatever it is.

      Sonal: And then all the things you can do to counter-program against that?

      Annie: Exactly, because that’s gonna reveal really different things. It’s gonna reveal some things that are just, sort of, luck, right? Let me think — can I do something to reduce the influence of luck there? Then there’s gonna be some things that have to do with your decisions. Like, I went into the break room every day and there were doughnuts there, so I couldn’t resist them. So now you can think about, how do I counter that, right? How can I bring other people into the process, and that kind of thing? 

      And then there’s stuff that’s just — you can figure out it’s just out of your control. It turned out I had a slow metabolism. And now what happens is that you’re just much less reactive, and you’re much more nimble, because you’ve gotten a whole view of the landscape. And you’ve gotten a view of the good part of the landscape and the bad part of the landscape. But I’m sure, as he’s told you, people are very loath to do these premortems, because they think that the imagining of failure feels so much like failure that people are like, “No, and you should, you know — positive visualization, and we should…”

      Sonal: I mean, the fact that in brainstorming meetings everyone’s like, “Don’t dump on an idea.” But the exact point is you don’t have to dump on an idea and kill the winnowing of options.

      Annie: No.

      Sonal: As part of the process you should be, then, premorteming it.

      Annie: Exactly. There’s wonderful research by Gabriele Oettingen that I really recommend that people see. The references are in my book. And across domains, what she’s found is that when people do this, sort of, positive fantasizing, the chances that they actually complete the goal are just lower <Interesting!> than if people do this negative fantasizing. And then there’s research that shows that when people do this time travel and this backwards thinking — that increases identifying reasons for success or failure by about 30%. You’re just more likely to see what’s in your way.

      So, for example, she did — one of the simple studies was she asked people who were in college, “Who do you have a crush on that you haven’t talked to yet?” She had one group who, you know, it was all positive fantasy. So, “I’m gonna meet them, and I’m gonna ask them out on a date, and it’s gonna be great. And then we’re gonna live happily ever after,” and whatever. And then she had another group that engaged in negative fantasizing. “What if I ask them out and they said no? Like, they said no and I was really embarrassed,” and so on, so forth. And then she revisited them, like, four months later to see which group had actually gone out on a date with the person that they had a crush on. And the ones that did the negative fantasizing were much more likely to have gone out on the date.

      Sonal: That’s fantastic.

      Annie: Yeah. So one of the things that I say is, like, look — when we’re in teams, to your point, we tend to, sort of, view people as naysayers, right? But we don’t want to think of them as downers. So, I suggest — divide those up into two processes. Have the group individually do a backcast. Have the group individually write a narrative about a premortem. And what that does is, when you’re now doing a premortem, it changes the rules of the game, where being a good team player is now actually identifying the ways that you fail.

      Sonal: I love what you said because it’s like having two modes as a way of getting into these two mindsets.

      Annie: Right, where you’re not stopping people from feeling like they’re a team player. And I think that that’s the issue, as you said. It’s like, don’t sit there and crap on my goal. Because what are they really saying? You’re not being a team player, so change the rules of the game.

      Sonal: You have this line in your book about how regret isn’t unproductive. The issue is that it comes after the fact, not before.

      Annie: So the one thing that I don’t want people to do is think about how they feel right after the outcome, because I think that then you’re gonna overweight regret. So you wanna think about regret before you make the decision. You have to get it within the right timeframe. What we wanna do instead is, right in the moment of the outcome, when you’re feeling really sad, you can stop and say, “Am I gonna care about this in a year?” 

      Think about yourself as a happiness stock. And so if we can, sort of, get that more 10,000-foot-view on our own happiness, and we think about ourselves as — we’re investing in our own happiness stock — we can get to that regret question a lot better. You don’t need to improve that much to get really big dividends. You make thousands of decisions a day. If you can get a little better at this stuff — if you can just, you know, de-bias a little bit, think more probabilistically — really, sort of, wrap your arms around uncertainty, to free yourself up from, sort of, the emotional impact of outcomes — a little bit is gonna have such a huge effect on your future decision making.

      Sonal: Well, that’s amazing, Annie. Thank you so much for joining the “a16z Podcast.”

      Mark: Thank you very much.

      Annie: Yes, thank you.

      • Annie Duke

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      So You Wanna Build a Software Company in Healthcare?

      Jorge Conde, Julie Yoo, and Hanne Winarsky

      Building a software company in healthcare is hard — and comes along with unique challenges no other entrepreneurs face. In this conversation, a16z bio general partner — and previous founder of genomics company Knome — Jorge Conde; and a16z bio partner and former founder Julie Yoo (of patient provider matching system, Kyruus) share their mistakes and hard earned lessons learned with a16z partner Hanne Winarsky.

      Why is this so damn hard? How should founders think about this space differently? What are the specific things that healthcare founders can do — when, where, and why? You’ll wish you only knew this when you started your own company!

      Show Notes

      • How the healthcare market presents a unique challenge to software companies [0:28], and details around issues they encounter [10:19]
      • Discussion around the best way to structure a software company in healthcare [15:53], including team-building [18:17] and product-building [23:23]
      • Understanding how slowly the healthcare space adopts new technology [25:34], and advice for functioning in a market that is distorted [31:06]

      Transcript

      Hanne: Hi, and welcome to the “a16z Podcast.” I’m Hanne, and this episode is all about “Building A Software Company in Healthcare.” In this conversation, Jorge Conde — a16z general partner in bio and healthcare, previous founder of the genomics company, Knome — and Julie Yoo, partner on the deal team for the bio fund, and previous founder of the patient-provider matching Kyruus — explain what it is that makes building a company in the healthcare space so fundamentally different from other sectors, and why exactly it’s so damn hard.

      Challenges of the healthcare space

      So, let’s start with, basically, just the very fundamental difference between building a software company, full stop — and building a software company in the healthcare space. What are the most foundational, crucial differences?

      Jorge: Well, historically, at least, software had two very important qualities in healthcare. The first one, the actual quality of software deployed in healthcare systems historically has not been great.

      Julie: User interface-wise and experience-wise.

      Hanne: Bad track record.

      Jorge: Bad track record there. And the second one is that it was usually not highly valued. So, at least a lot of times it was considered either free or cheap.

      Hanne: And why was that — that from the very beginning, there was not a lot of value attached to this?

      Jorge: In the healthcare system, a lot of things still have a very human component to them. Automating things and, sort of, creating frictionless experiences or delightful experiences — the things that software is really good at doing — is just really hard to do in the healthcare system. The second one is, and I’m gonna generalize for a second — but I think a lot of times in the healthcare system, software is sold as a component of a broader service or of a broader offering. And so, therefore, it’s the piece that tends to get, sort of, devalued first, because it obviously has the lowest marginal cost. It’s kind of created this weird dynamic for software companies that are trying to build in healthcare.

      Julie: There’s a higher degree of sensitivity in this particular market for things that get in the way of the patient-provider experience. One of the challenges/opportunities within healthcare is that it tends to be much more risk-averse when it comes to adoption of new technologies. One meaningful difference in introducing a software product to this market versus other markets is the level of scrutiny, and the bar that you need to hit from a, you know — not even usability perspective, but just utility, and actually having validation of — if you are going to introduce something new into the care delivery flow, it better work, because the stakes are so high, right? If you get it wrong, you could either send a patient in the wrong direction, or they might not get the care that they need, or it could actually harm the individuals involved.

      Hanne: So, not just higher barrier to entry, but higher stakes immediately?

      Julie: Correct.

      Jorge: And you have a reticent buyer, generally speaking. They’re running on very thin margins. If we’re, like, selling into the healthcare system, the provider space, it needs to work. Because if it doesn’t, obviously, there can be patient harm. And so, you know, the probability that a newcomer, an upstart, can come in and sort of make that case in a convincing way is a very, very difficult challenge.

      Hanne: So does that mean you have to have certain prerequisites that you may not need to have in other spaces? If you know you have these challenges and you know that you’re entering this space with a lot more barrier to entry and a lot higher stakes, like, are there certain things you need in place, you know — a certain kind of proof of concept that you might not have to have otherwise?

      Jorge: Well, first of all, I think you’re touching on a very important thing, which is, in the space — and I’m going to specifically focus on, sort of, the healthcare system. So, let’s call it provider systems — payers, and the like. You have to really understand what the workflows are, what the problem space is, and how to actually address any of those things. And so, one of the biggest challenges, I think, that companies have when they wanna build software products here, is to really understand what problem they’re gonna solve. Because I think you have this weird sort of intersection between — it’s very non-intuitive, it’s still very human-driven and centric, there are regulatory barriers. You don’t wanna get in between, say, a provider and a patient. You know, most people aren’t born with the ability to say, like, “I know I can insert a piece of software into this part of the workflow, and I will solve an acute pain point for the system.” That’s not obvious.

      Julie: Yeah. And some of that is actually lack of standardization. You would think that medicine is an industry that has a tremendous amount of standardization and protocols around how people make decisions and do things, but it actually turns out that healthcare is an industry that actually is characterized by a tremendous amount of variation.

      Hanne: And variation in what kinds of ways?

      Julie: It could be variation in terms of actually, literally, the decision that if you have 10 doctors who are all presented with the same patient, you might see 10 different decisions about how to treat that patient. Some physicians might be more aggressive about using invasive surgical techniques, versus others who are more holistic. Even just how I was brought up, religiously or culturally, might impact the way I think about that problem. From a product perspective, you could have multiple drugs that all treat the same condition that all have different implications and whatnot. So, even there, even though you have a patient population that is characterized by the same diagnosis, you could have dozens of different ways that those patients play out. And so it makes it very hard for a technology company to come in and, sort of, generalize, and say, “There is one single method for manufacturing this thing or for making this decision and managing this patient population.” Ultimately, that reflects as differences in the financial profile of different patients. 

      Jorge: Healthcare, it’s like politics. It’s very local. Thinking that you’re gonna have an out-of-the-box, one-and-done solution — even in systems that look similar from either a size standpoint, or a reach standpoint, or even a geographic standpoint — these are all kind of “n of 1s”.

      Hanne: So, what does that mean? So, we have, kind of, knowledge of workflow, the knowledge of variety and spectrum, and that you are ultimately working in, weirdly, an “n equals one” scenario. I wanna bring it back to like actual practicalities of this sort of company building. In your experiences, you both founded companies — what do you wish you had known or done differently from the very beginning, given the complexity of that space, and the unique challenges that building a company in healthcare presents?

      Julie: With Kyruus, one of the products that we had was a product that was used by call center agents in hospitals. And our thesis when we first launched the product was, “Oh, well, we’re just gonna go after every hospital that has a call center, and they probably all operate similarly. And what constitutes the job of a call center agent is probably relatively homogenous. And so we can make all sorts of assumptions about how it’s built, how it’s deployed, and how it’s managed over time.

      Hanne: The thing that strikes me already is that feels like a reasonable assessment of the lay of the land.

      Julie: Yeah. And especially, I think it’s very easy to get fooled in healthcare by looking at other industries and seeing how it works in the rest of the world, because certainly…

      Hanne: And then you pull up the…

      Julie: Yeah. And then you pull up the wool and it’s like, “Oh, it’s completely opposite.”

      Hanne: Yeah, it’s something else completely.

      Julie: Call centers, I mean, that’s definitely an industry that if you look at retail, or even all the airline companies and how they operate their customer service operations — tend to be pretty standardized, and pretty sophisticated, in a lot of cases.

      Hanne: When did you start to realize this wasn’t maybe, like, your average call center?

      Julie: Like, on day one. First of all, there’s heterogeneity in the actual scope of services of pretty much every call center that we encountered. Some call centers might be fully centralized, and they’re, like, a central 800 number that receives every call that comes into the hospital — versus others that are decentralized, that only serve the primary care line, versus the cardiology line, versus the dermatology line. And because of that, they will have just fundamentally different starting points of where they have to be in the workflow for the thing to work.

      The other aspect is the scope of functions that the call center plays. It could be everything from just a general marketing service, where a customer might call in and say, “Do you provide these kinds of services? Can you give me directions to the clinic?” All the way to, “I need a prescription refill. I’ve been diagnosed with this thing, I need to figure out what kind of surgery I need.”

      Hanne: So, again, a much bigger range of possibilities, basically?

      Julie: Correct, yeah, when you boil that down to, like — I’m a call center agent, and how do you define my job so that when I give you another piece of software to use to do that job, it’s gonna be seamless? And when you have that kind of heterogeneity around even the sheer definition of what the job is, it makes it very hard to design a scalable solution that can, kind of, fit into all those different environments.

      So, day one, we actually were fortunate to get a customer that did have a pretty robust centralized call center group that was hundreds of people, who literally were answering every call that was coming into the health system. And so, the immediate sort of leap that we made was, “Oh, they must all look like this. Even if 80% of it was the same, and there was a 20%, sort of, buffer that needed to be modified, we can deal with that.” Yes, they all had central call centers, but the fundamental scope of jobs that they were doing were completely different across the board. And some were more clinical in nature, some were more marketing in nature, some were more financial in nature, etc.

      Hanne: So, what were the knock-on effects of that?

      Julie: Yeah, it probably had an impact on, like, go to market, product design, and product strategy. Most importantly, the service model of — you could either say, “We’re gonna design our software to be so flexible that it could work in any environment.” Or you could say, “We’re gonna provide services to come train your people to behave in a more standardized way, relative to the rest of our book of business.” And so we ultimately ended up taking a hybrid approach to both. But the latter, you know — that services approach — is something that we hadn’t thought about, that allowed us to sort of abstract out the variation to some degree, but also provide value back to the customers in a pretty unique way. Because then we had the best practices for how it should work.

      Hanne: So, ultimately, it was a good thing — but it was a major fork in the road, basically?

      Julie: Absolutely. Because there’s so much variability, because there’s so much localization, the notion of the pure SaaS model — where you’re just throwing technology over the fence and assuming that it will fit into whatever environment you’re deploying it into — that is a moot point in healthcare. You actually do need to think about the services component of things. There was a whole generation of companies that got started, like, a decade ago that took these sort of tech-only approaches and failed to get scale, or had to fundamentally pivot their models to actually take into account more of the human element of the service delivery model.

      I mean, even — there’s a term for it now, right? Tech-enabled services is a way of doing things now in digital health that, I think, is well recognized that it’s necessary to wrap the technology with a human component to essentially address and be able to accommodate all the variation that you see across different customer bases. And it changes your cost structure fundamentally. The nature of how we talked about the business and how it scales, and even our fundraising strategy, fundamentally changed because of that. And so we did have to raise more, and give ourselves more runway, and think about different ways to manage our margin.

      Hanne: It sounds like everything that could have been changed was changed by that.

      Julie: Yeah.

      Specific issues encountered

      Hanne: Let’s go back to a specific example where you really put your foot on it.

      Jorge: Well, so — our experience at Knome was interesting, because here this is a company — the sole purpose of the company was to provide software capability to analyze genomic information. And so when you launch that, your assumption is, “Well, this could be used to power all kinds of applications.” It could be used for research, either in academia and industry, it can be used for clinical diagnostics.

      Hanne: It’s flexible.

      Jorge: We thought it was very flexible. And so challenge one is, you know — a solution looking for a problem is always a very, very dangerous thing. I think that’s universally true, and it’s especially true in the healthcare space. And challenge two was understanding exactly where, in the case of the clinical setting, where this technology would be used in the workflow. So, here, we wanted to go after the clinical labs.

      Hanne: That was your initial hypothesis?

      Jorge: Our initial hypothesis for an application in a clinical setting — you have technicians and docs that are inside of the laboratory setting receiving samples, running a test, analyzing the results of that test, generating a report that gets signed off by a lab director that goes back to a physician. Usually it’s in the form of a diagnosis, right? And it gets signed off and it goes to the physician. The physician now takes that report, and basically, decides what to do based on that information.

      So, our assumption was, “Well, if you have the ability to sequence DNA now, in a way that you couldn’t before. Before, you’d have to do all of these specific tests, you have to know what to test, and then you’d test it, and then you’d get a report. You had to know what streetlamp the keys were under, right? Like, there in that case. Whereas once you had the full genome, you would just sequence everything and just run a bunch of software queries. So, our thought going into this was, “Well, that’s an incredibly powerful tool for clinical labs. Because first of all, you can sequence just once and analyze over time. So, you can again and…

      Hanne: Right. Again, it seems like a totally legitimate assumption to make.

      Jorge: Right? And it turns out that there was a lot of challenges with that assumption. The first one is, every lab is different. A lot of them didn’t have the budget, or the willingness, to basically pay the upfront piece to buy the capability to use this technology — or they didn’t have the ability to sequence everything upfront, even if all of the subsequent queries would be technically free later.

      Hanne: Why not?

      Jorge: It’s the way they’re reimbursed.

      Hanne: Oh, how fascinating. Too expensive, basically.

      Jorge: It’s too expensive. So, even though theoretically there’s an ROI — a return on the investment of sequencing upfront — just the way the industry is structured, the way reimbursement flows, the way payments flow, it just didn’t make sense for a lot of labs to do this.

      Hanne: So, how was that not just a complete roadblock at that point?

      Jorge: It was a big roadblock. So what that required us to do was to then focus on clinical labs that had the ability to make certain investments in upfront cost. And those tended to be very sophisticated labs that do a lot of research work, in addition to patient care. And they tended to be on the bleeding edge, and they wanted to incorporate new technology, and they were great partners and all of that. But then it goes back to your “n of one” problem. 

      So, you sell something into that lab, and you go next door, and next door has a totally different set of capabilities — a totally different set of constraints, a totally different set of expectations. And so, therefore, all of a sudden, the solution you created for lab A is not relevant, or unattainable, for lab B. Now, to just add to stepping in it — you know, when you’re analyzing genomic data, there’s a massive amount of computation required. And so we went in there assuming, “Well, this is easy. We’re just gonna shoot all of this up to the cloud, we’ll run the analysis, we’ll send the data back to the lab, the lab could verify it, generate a report, and off we go.” It turns out labs weren’t comfortable sending data up into the cloud, full stop.

      Hanne: At that time, it was just completely…

      Jorge: At that time. Arguably, even today. Arguably, even today in 2019. But definitely, at that time, we probably should have known that earlier, that would have changed how we thought about going into the clinical lab space.

      Hanne: How would you have done your homework? I mean, what would that have actually looked like?

      Jorge: It was frankly, I think, just defining the specs of what would be required to bring in our technology. Because I think people intuitively know that genomic data is massive, but I don’t think they know the level of computation required to run the interpretation.

      Hanne: Right, so like really running the numbers.

      Jorge: Running the numbers for them. And by the way, we tried everything. I mean, we brought representatives from AWS that could show them that they had a HIPAA-compliant cloud, that they had received all the certifications, and it came back to risk aversion. So, someone —the lab director — saying like, “Look, I’m sure all of that’s true, but I’m not gonna risk sending all of this data up into the cloud.” So, that was a big, big challenge for us, and it ended up being a major limitation for our ability to expand into the clinical setting, because of all of those barriers.

      Hanne: So, what did you do?

      Jorge: We had to do a plan A and a plan B. And so the plan A was, we assumed that there would be a couple of forward-looking labs, or forward-thinking labs that would be willing to work in a cloud environment. Much easier to deploy there. The plan B was, we had to create a box. We had to create a box, and the box had to have, essentially, the computational capability.

      Hanne: A Knome appliance?

      Jorge: Yeah, we had a Knome appliance.

      Hanne: Yeah, I remember that. Oh, my gosh.

      Jorge: Because they didn’t want the data to go outside. And it’s for the reasons that we’d expect. You know, there’s regulatory, there’s risk associated with that today in 2019. In fact, the companies that have managed to use this technology have taken the sort of full-stack service approach. So, that sort of high-low strategy became the approach, is — get folks to deploy into the cloud, when they were willing to. And in the case where folks needed an appliance, we basically had to go to labs that had enough sample volume that an appliance made sense for them, and make, basically, the case there from an investment standpoint.

      Hanne: So, again, multiple-choice, variety, and addressing in different ways.

      Structuring software companies

      Jorge: A pure software company in healthcare is a really hard thing to do. Because on the one side, you have this challenge that — it’s hard to create a solution that’s gonna fit everyone. And, therefore, you need to have some level of services around that software. That’s on one extreme, so you need to have humans in the process, or in the loop. And then the other extreme of it — if it is pure software, then it’s considered that it should be free, so it’s very hard to abstract value.

      Hanne: That’s so interesting. Do you think that’s shifting at all, with the, kind of, understanding of the importance of data and some other things?

      Jorge: Yeah, look, I would argue it’s shifting on a couple of axes. The first one is — is data is becoming more and more valuable. Historically, data was viewed as being either too small in terms of its impact, too narrow, too dirty, etc, etc.

      Hanne: Too difficult.

      Jorge: Yeah, too unstructured. So, that historically has been the case. So, if you have ways to ingest data and clean it and make it meaningful, then I think that is valued. Probably the most public one is what Flatiron was able to do, and ultimately getting acquired by Roche for $2 billion. That’s viewed as using an electronic medical record to capture patient experiences, take that information and give researchers the ability to drive valuable insights from that. That’s a relatively new thing. So, I think there is the ability to create value there. So, I think that’s one axis. 

      I think there’s a general shift in the model that having a tech-enabled service can be a valuable thing, and if done well, can be a scalable business. In other words, if you know what you’re trying to build, and if the software layer reduces sufficient friction in the system and allows you to add people — not linearly, as you scale, right, but in a leverageable way — then all of a sudden, you could have tech enabled services that can grow and become large businesses.

      Hanne: So, leaning into what it is that makes it difficult almost, and then scaling that, leveraging that.

      Jorge: Exactly, finding ways to make that scalable. That’s not easy to do, but I think it is now doable in a way that probably wasn’t a decade ago.

      Julie: And I think we see that same trend actually happening in the consumer world, where you used to have a bunch of services, like, the marketplaces that were purely tech, and were just matching supply and demand and then getting out of the way. Whereas now, you see a lot more services, like, in the real estate market, where they’re actually managing properties — or actually gonna clean the place and make sure it has good furniture, and all that kind of stuff. I think the same premise holds true in healthcare, where you realize that in order to truly make an impact, you kind of have to own certain parts of the full stack. And that’s what you see playing out in the rest of the world as well.

      Hanne: Okay, so we’ve talked about kind of knowing the workflow and the complexity of the system, running the numbers and speccing it out as concretely as possible. How about in terms of team building? Are there ways that you, knowing what you knew down the road, that you would have changed how you thought about building the team from the very beginning?

      Julie: My prior experience was not in healthcare, and so a lot of my views on how to do these kinds of things were informed by a company that was just a pure enterprise software company. And one of the mantras was — you wanna, in the early stage of the company hire for all-around athletes, and just people who are utility players. Who can, like, roll with the punches and figure it out. It doesn’t matter what kind of experience they had, as long as they’re scrappy, intellectually motivated people, they’re gonna figure it out. So, [we] certainly took that approach when we started Kyruus and hired folks — not necessarily from healthcare, who maybe had some engineering experience or sales experience from elsewhere in the world and said, “We’re just gonna go in there and figure it out.”

      Hanne: But you surely had some deep experts in the space as well, no?

      Julie: So, my co-founder is a physician by training, so we had, sort of, the deep clinical knowledge. But I would say, actually, we didn’t have that many people who knew the specific market that we were going after. And that’s another characteristic of healthcare startups is — healthcare is so massive, that when you talk about market segment, you have to be very specific about what you’re talking about. So, like when people come and say, like, “Oh, I have a company that sells to providers.” I’m like, “That’s great. That’s like, you know, 20 billion .”

      Hanne: What does that actually mean?

      Julie: Yeah, like, “There’s 20 billion ways that you could just describe providers. Like are you selling to hospitals, are you selling to health systems, are you selling to individual practices? And each of those can be multibillion-dollar markets in and of themselves.

      Hanne: I used to work in publishing, and it reminds me of people who would pitch their books to us and be like, “It’s for the general reader.” <laughter> There is no general reader. There’s, like, somebody who likes to read Amy Tan, and there’s somebody who likes to read, like, Dan Brown, or whatever. Like, these are different people.

      Julie: That’s a good analogy. Yeah, there you go. So, yeah, so basically, we had folks in our company who had “healthcare experience,” but maybe it was from the pharma industry, or from payer, <Not super relevant, yeah.> or even like a different segment of the provider market, but not the specific market that we were going after, which was, like, a very esoteric — we were going after the biggest health systems, like, the top-down approach in the enterprise space. And there’s very specific characteristics of those organizations that are very different than even smaller hospital networks.

      The areas of the team building exercise that I wish we had been more thoughtful about were, in terms of customer-facing roles, where it was the team responsible for managing the customer relationship longer term — you know, just how important it is for those people to have some kind of understanding and empathy, and ideally, experience, with the kind of people that we were servicing. There is total merit to saying, “Actually, we need some insiders who might not have any technical skills whatsoever, but can help us understand the culture, and the politics, and what it means to even, like, talk to a physician.” We had a bunch of folks who had never been in healthcare, who walked into meetings and called doctors by their first names. And that was a complete taboo in certain cultures, where you have to call them Dr. Jones or Dr. Smith.

      Hanne: Like, “Stranger in a Strange Land.” Here’s the language here.

      Julie: Yeah. So, I think from a team building experience, one of the biggest lessons that we certainly learned was A, valuing healthcare domain expertise earlier in the evolution of a company relative to other sectors. And then, also thinking about where that makes sense — like, what functions that makes sense, because it’s not [a] 100% universal statement across the board. I would say, our engineering team — it was actually better that they came from outside of healthcare because…

      Hanne: Oh, so in specific areas where you need knowledge and where you don’t. That’s interesting. Why was it a bad thing for engineers to have that?

      Julie: Not a bad thing, per se, but you wanted people who could, like, really think out of the box, and not be, sort of, married to the way it’s done today, because actually, that’s exactly the point of building companies in this space is to not do it the way it’s been done. And so most of the technology systems that are in place are written on super legacy technologies, and don’t have things like APIs, and whatnot. You need to be super creative about how to get into these systems and get data out, because they were fundamentally not designed to have liquidity around the data that’s stored in them.

      And so, it was helpful to have people from the financial services industry, for instance, who had figured those things out, with similar banking systems and whatnot, and could kind of bring some of that creativity to the healthcare space. So, engineering is definitely a space where I felt there was a positive to not having that healthcare domain knowledge. But certainly on the commercial side of the business, I think it’s critically important.

      Jorge: Making sure that the engineering team is as modern as possible is the most valuable thing you can do for your company. Because I think what’s generally true, and probably definitely true across the board, is in healthcare, the data sets are so complex, right? They’re complex in terms of their variety, they’re complex in terms of their volume, they’re unstructured, there’s regulatory requirements. There are so many things that are challenging from a data-handling standpoint, so building the pipes in the most modern way possible — absolutely critical. Whoever is customer-facing, I think, has to be from that game, has to understand the space, has to understand who the customer is, has to understand the cultural norms and all of those things. Those things are both true.

      Hanne: So you need both from the get go, immediately.

      Jorge: You need both from the get go. Industry-specific on the customer-facing side and domain expert from the engineering side, right? And then let’s talk a little bit about the middle — the product, right? That’s where the sausage gets made.

      Julie: Totally. I’m gonna be biased, because I was the Chief Product Officer of my company, and that’s where I would say it was split — where I do think it’s important for the leader of that organization to have a pretty deep understanding of the market. And so, I happen to have had healthcare experience — not specifically in this particular segment, per se, but I understood some of those cultural nuances and just dynamics of how the market worked to be able to set strategy.

      Below me, however, some of my best product managers were not healthcare people at all. And, in fact, we had three products — one that was the call center product that I mentioned earlier, where the end users themselves were not healthcare people, right? And so these guys, you know, some of them were, like, high school graduates who go home and they use their iPhone, and they’re used to all these modern technologies in the rest of their lives — and then they come to work, and they’re faced with these totally esoteric, crappy hard-to-use systems. And so I wanted someone who actually had, kind of, a consumer mindset.

      Hanne: Did you find yourself doing a lot of sort of explaining and educating, though, to bridge that gap?

      Julie: Yeah, my philosophy was just throw them in the deep end. As part of the onboarding experience at Kyruus, you had to visit a hospital call center, and they actually let you listen in on calls. There was, like, a religious transformation for these team members who went. Some came back and said, “I cannot believe that this is how these organizations operate.” Because everyone thinks of healthcare as this very pristine, like — “I’m going to trust you with my life.” And they’ll come back and be horrified, because they see that things are being run on paper and just how much burden they put on the customer. 

      Because part of what you hear when you’re listening in on these calls is, like, asking the patient, “What do you wanna do?” And the patient’s like, “Well, would I have understanding — I’m calling you guys, the hospital. You’re supposed to tell me what to do.” So that was one reaction. The other reaction was completely emotional, because a lot of these patients who were calling in had just been diagnosed with cancer, and they have no idea what they’re doing, and they’re calling because they need help. And then the call center agent sometimes felt helpless, because they didn’t have the tools or the workflows or the information.

      Hanne: Oh, it reminds me of, like, a 911 operator with no training. Somebody’s thrown into the middle of, like, I’m having a massive life crisis.

      Julie: Exactly. Yeah, it was inspiring and motivational, and so that became part of our training process — was to just go out there and see it versus me explaining it.

      Getting the timing right

      Hanne: That’s really interesting. Okay, so what about timing? Do you think it’s different in the healthcare space, how you think about — what’s the right moment for your product?

      Jorge: One of the big challenges in healthcare is this idea that you can be too early. You can be too early for a couple of reasons. One is, you need a lot of changes to workflows for the entire system to become much more modern.

      Hanne: But you think this is different from being too early with, like, pets.com?

      Jorge: That’s a good question. So, look, the way I would think about it — I described what was, for us at the company, a very obvious evolution of where genetic testing would go. You would sequence everything first, and you would test multiple times in silico.

      Hanne: You could see the light at the end of the tunnel.

      Jorge: I mean, that’s a clear future. And so the question is, when is the system ready for your particular solution to a problem that everyone agrees exists, right? Everyone agrees that we have to do a better job at being able to diagnose folks with genetic disease. And I think everyone would agree that using genomics, the ability to do this at large scale, to query multiple times, to use software to make intelligent queries — would be a very powerful tool, a very powerful solution for that. But the reality was — continues to be — that just the structures of the industry are such, even though that’s where I think we will end up, it’s just not ready for it now. And I think this is true for any entrepreneur, right? Timing is a big part of anything you do. I think timelines are especially warped in healthcare, because it just takes a long time to adopt new technologies.

      Julie: There actually is a peer-reviewed study of the average number of years it takes for new technologies that are introduced into the medical setting to become mass-market adopted, and it was…

      Hanne: Fascinating. Wait, wait, let’s guess — two years?

      Julie: Seventeen years.

      Hanne: No!

      Julie: Yeah.

      Jorge: Well, I mean, we still have fax machines.

      Julie: Yeah, we still have fax machines, we still use the same…

      Hanne: That’s true. But we’re not talking about when technology leaves. But you’re right. It’s the same thing really. Yeah, when it gets <crosstalk>.

      Julie: So you can think about it as all the things that have tried to replace the fax machine are not yet mass-market adopted. And it’s the same — you could see it in — I think the study actually focused primarily on, like, stethoscopes and thermometers, and things that literally have not been redesigned for hundreds of years, because it’s been so hard to disrupt them.

      Hanne: Yeah, over the last 17 years, there’s been a bajillion better versions of the stethoscope that we’re just not seeing. The wheel could have been reinvented, but better.

      Julie: Absolutely. Those are the tangible examples, but the same applies to software and technology, and that’s a lot of the reason why you see the market-leading companies that own the EHR space today are literally 45 years old. And by the way, those companies also didn’t hit their stride until, like, 20 years into their journeys, right?

      Hanne: So, time functions completely differently, basically, in this system. It’s almost like…

      Jorge: It’s like a wormhole.

      Hanne: And second of all, it’s [an] incredible testament to the strength of these systems that…

      Julie: Totally. It’s like, once you do make it, it’s totally sticky. The LTV, essentially, of tech companies that actually make it and get to a certain level of scale is through the roof. There’s no incentive to rip them out because if they work, they work. The switching costs, because of all the human and cultural elements that we described, is huge.

      Hanne: Yeah, so the longevity of your company, if you’re looking at success, is also incredibly promising.

      Julie: Yeah. I mean, certainly at Kyruus, the way we mitigated it was we thought about what our fundraising strategy would be to give ourselves enough runway to have that model play out. We needed to fund the sales cycles and the adoption cycles to create a new category of solution that didn’t exist.

      Hanne: Just to hang out in the wormhole for a while.

      Jorge: It’s a big oxygen tank.

      Julie: Yes, nothing meaningful happens in healthcare in under three years. And so you kind of have to give it some runway.

      Jorge: That’s one of the things that we’ve spent time talking about is, what does a minimum viable product in healthcare look like?

      Julie: Yeah, it doesn’t exist.

      Jorge: Big gang, you’ve got to go in, and you’ve gotta create a category and you gotta get that adopted.

      Julie: I think in other industries, you can sort of “get away” with having a product that does one thing really, really well, and then start there — and, yes, expand over time. But at least you can get buy-in to prove your value with that initial use case. I think going back to one of the points you made earlier, in healthcare, when you’re in the flow of impacting a patient encounter, and saying, like, you’re gonna rip something out or change the way that you’re doing something or what have you, you have to make sure that it’s gonna give you the right answer, so to speak. And so even if it’s just one feature, it might mean — okay, yes, it could be one feature, but you have to be integrated into seven different systems to make sure that the data flowing into that one feature is enough to inform the right outcome or decision…

      Hanne: So really fully baked.

      Julie: If the transaction falls through the cracks while you’re doing some kind of revenue cycle type encounter, you might not get paid for a procedure that could have a severe impact on your bottom line. You need more funding, you need to think differently about your strategy for product and what that footprint looks like.

      Jorge: You have to have the full solution. And the related point I would make to that is, it’s really hard to have a point solution, even if that point solution is very, very good. I think people in general in the healthcare system are looking to buy a complete solution. So, if you take the problem from A to B to C to D, that’s great. But somebody — they need A to Z. And if they can’t get A to Z from you, it’s very hard to get them to buy A to C from you. I’ll go even further than Julie — I will say not only does MVP not exist in healthcare, I would argue that product market fit doesn’t exist in healthcare.

      Hanne: What do you mean by that?

      Jorge: You know, the definition of product market fit is, when the right product meets a good market, right? All of the things we talked about create such distortions in the marketplace that by the time you actually get through all the hoops, you have such a, sort of, skewed product. It’s not really product market fit, it’s almost, like, accepted product capture. Here, you have regulatory issues, you have pricing concerns, you have incumbents. You have so many aspects that sort of distort the market, that I would argue that you don’t have a normally functioning market for software in healthcare.

      Working in a distorted market

      Hanne: How would you both embrace that distortion early on, and not get completely sort of knocked off your path by it? Because it strikes me that a lot of what you’re describing is, kind of, like — know thyself. Like, know yourself very deeply. And, like…

      Julie: That was the tagline at Knome, by the way.

      Jorge: That was Knome’s tagline. Know thyself.

      Hanne: Oh, was it really? That’s really funny. I did not work that in for you. But also, like, know where you’re going, and do that, kind of, deep — I wanna say, like, soul searching on a company level and build out accordingly. So, how do you get that big center of gravity of really knowing yourself, knowing where you’re going, but be able to be flexible with that distortion along the way?

      Jorge: The only North Star you can have — and this is gonna sound cliche — but really understanding your value proposition truly, from the customer standpoint, becomes a critical sort of guide for what you do. And this is a debate that healthcare companies have all the time, which is, should your value proposition be, “I’m gonna save the system money?” Because the healthcare system is very inefficient, and it runs on very low margins, generally. Should it be that “I am gonna result in better outcomes for patients?” Is it gonna be, “I’m gonna create some sort of lift in terms of return on investment?”

      There’s a bunch of different ways you can think about value proposition. If you don’t have that crystal clear from the outset, the amount of obstacles that you are going to hit along the way are gonna make it such that it’s gonna be very difficult to get to the other side. If you don’t really understand the workflow, and the culture, and the regulation, and the governance, and the politics, and all of the other things, you can have a theory on what the value proposition is, but you need your customer to confirm that early on. And sadly, the best way to confirm that is to have them buy something, obviously.

      Julie and I have had this debate before, which is — a lot of the software platforms that go into healthcare have been sort of predicated on, “We’re gonna cut costs.” And I don’t know of any, sort of, solution out there that has meaningfully been able to make a very, very strong case that they can cut costs. And by the way, part of it is, I think — is because it’s really hard to measure costs.

      Julie: It’s almost, like, a necessary evil where you have to say — in some way, shape or form — you are gonna reduce costs, but that can’t be your primary value proposition. Because at the end of the day, it’s a line in the cost structure that can get wiped out over time and potentially get commoditized.

      Hanne: So, is the takeaway — know your value proposition as early as possible and test it?

      Julie: That, and then have the conversation of like, “Okay, if we were able to accomplish what we just described, is it worth it? Is the juice worth the squeeze? Because it’s so expensive to distribute product in this market, because of the sales cycles and the nature of the enterprise sales motion, and whatnot, that if you’re not able to envision a path towards being, like, at least a $500,000 kind of a year type solution in this space, it’s actually not financially worth it to build a business in that area.

      Hanne: Right, which goes back to your point of, like — run the numbers, basically.

      Julie: At least, like, back of the envelope, whiteboard kind of thing.

      Hanne: Yeah. I mean, is there anything that you can figure out as you go? It sounds like you need to know so much before you begin, and be so self-aware, and so, kind of — have the end game in sight. Like, are there things that you can leave, sort of, more organic, and feel out as you go?

      Julie: Yeah, no, I mean, absolutely. There are tons of things you can be doing on a daily basis with end users. And just feedback mechanisms on, like, how people — are they actually able to do their jobs, for instance, and making minor tweaks to the workflows and whatnot. So, that was always a component of a more organic and dynamic aspect of how we did things.

      The other thing that you need to, kind of, think about doing in parallel, is — so much of success of technology in healthcare is predicated on integrating into other ecosystem players. And so this is actually — probably one industry where you definitely can’t just build in a vacuum. You actually should understand, even if it’s not for another few years that you’re really gonna have to do this, like — who are the players we just need to get to know. So that we’re on their radar when time comes for us to take the hammer, and try to break down the wall of integration with that vendor — that we are on their good side and that they know who we are so we can kind of make that happen faster. So things like that, I think you can be doing in parallel to the kind of formulation of what the footprint of the product is. 

      Jorge: If you’ve got the right solution, you can get very creative in how you get paid. So figuring out different pricing structures or value capture mechanisms, I think, is something that you can do pretty organically. Because, if you are making a difference in the system, the system has so much cost built into it, and so much revenue flowing through it, that there are ways to be very imaginative there. So, that’s the first thing I would say. The second thing I would say is, thinking about adjacencies — going from one — your core function, to the next adjacent use case. Not all adjacencies are created equal. One might be easier than the other. It’s almost like jumping on stones across a pond or something, right? What’s the next one I can jump on that’s least likely to make me fall into the water, even if it doesn’t get me as far as another one?

      Hanne: Yeah. Right, always have that “closer spot” insight.

      Jorge: Yeah. Because you’re creating the next thing and the next thing and the next thing, and you build up from there. And eventually, you cover so much surface area that you become a very sticky solution and you, hopefully, become a complete solution, sort of, closer to the A to Z type vision.

      Hanne: Okay, last question. The biggest takeaways? Quick lightning round for your founder struggling right now, what would you say? Bullet points.

      Julie: Know your market segment. Be very specific about what segment you’re going after, because that has major implications for your go-to-market and your product.

      Hanne: Good one. Jorge, biggest “you wish somebody had said to you?”

      Jorge: One is build the multidisciplinary team early. Two is understanding if the person that suffers from the pain point can actually pay for your solution, because there’s a lot of misaligned incentives in the healthcare system. And three, with the right technology, you can have massive impact on patient lives and the experience that we have with the healthcare system — which we will all touch in our lifetime. And if there’s anything you can do to make it better as an entrepreneur, I would say that is extraordinarily satisfying.

      Hanne: That’s fantastic. Those are some good bullets. Thank you both so much for joining us on the “a16z Podcast.”

      Julie: Thank you.

      Jorge: Thank you.

      • Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

      • Julie Yoo is a general partner at a16z where she invests in healthcare technology. Prior to joining the firm, she was the cofounder and Chief Product Officer at Kyruus and VP Product at Generation Health.

      • Hanne Winarsky

      Inside Apple Software Design

      Ken Kocienda and Frank Chen

      Join a16z Deal and Research operating partner Frank Chen for a conversation with longtime Apple software engineer Ken Kocienda for an insider’s account of how Apple designed software in the golden age of Steve Jobs, spanning products like the first release of Safari on MacOS to the first few releases of the iPhone and iOS (very first codename: “Purple”).

      In this podcast, which originally appeared as a video on our YouTube channel, Ken vividly shares about the creative process, how teams were organized, what it was like demo’ing for Steve Jobs, and many other fun stories. Along the way, we repeatedly probe the question: is Apple’s obsession with secrecy during the product development process a feature or a bug?

      Show Notes

      • Background prior to joining Apple [0:00] and early work designing the Safari browser [6:26]
      • The influence of Steve Jobs on the creative process [15:03]
      • Short stint in management and transfer to the iPhone design team [20:54]
      • How the iPhone team was structured [29:13] and technical issues building a touch keyboard [37:00]
      • Apple’s culture of secrecy and the difficulty of getting outside help [43:19]
      • What it was like to demo for Steve Jobs [54:06], and further discussion about his influence and personality [1:09:21]
      • Why having a liberal arts background is beneficial [1:18:07], and general advice for young people interested in tech [1:24:32]
      • Discussion of Apple’s culture of collaboration [1:30:30]

      Transcript

      Frank: Hi, welcome to the “a16z Podcast.” This is Frank Chen. This episode, which is called “Inside the Apple Software Factory,” originally aired as a YouTube video. You can watch all of our YouTube videos at youtube.com/a16zvideos. Hope you enjoy.

      Well, welcome to the a16z YouTube channel. I’m Frank Chen, and today I am so excited. I feel like I have won the golden ticket to Willy Wonka’s Chocolate Factory. Because look, if you’re in Silicon Valley, the one chocolate factory you want — you’re desperate to go visit — is Apple. And the reason for that is Apple has consistently, over its history, turned out some of the most intuitive, and delightful, and just plain awesome products that people use. And people are dying to find out — how is it that Apple makes such delightful products? And so today, I’m here with Ken Kocienda. And I’m so excited for him to tell us all about the creative process that he used, and his team used, to create these products. So Ken, thank you so much for coming.

      Ken: Well, thank you so much. It’s great to be here with you.

      Frank: Well, let’s get right into it. So maybe talk a little bit about how you ended up at Apple. Because, like, on paper, you don’t look like the typical software engineer. So go back and do the long — like where were you born and, like…

      Ken: Well, I was born in New York, stayed there on Long Island, downstate. Grew up close to beaches. Lived there until I went away to college. I went to Yale and got a degree in history. And then after I graduated from Yale, I didn’t do the typical thing, I went to motorcycle mechanic school.

      Frank: Really?

      Ken: Mm-hmm.

      Frank: All right. Ivy League to — and what motivated that, like, you just loved motorcycles?

      Ken: I wanted to learn how to fix motorcycles. When I graduated from college, I wanted to do something that was as different from Ivy League college as possible. This…

      Frank: I think that qualifies, motorcycle maintenance.

      Ken: Right. This was dismaying to my parents — my father, in particular, I can tell you.

      Frank: I’m sure.

      Ken: But yeah so I…

      Frank: At least you didn’t have an Asian parent.

      Ken: Well, I think…

      Frank: You would have been disowned, that’s like…

      Ken: …my dad was pretty confused about the choice. But anyway, But eventually, you know, they got behind and supported that. And so I fixed motorcycles. And then I wasn’t really quite sure what I wanted to do. I had this degree in history, but I wanted to, you know — kind of, keep following my nose, finding new and interesting things to do. I also did a lot of work in photography when I was at Yale. I spent a lot of time in the art and architecture library on the Yale campus, just reading books, learning about art.

      Frank: Beautiful buildings on campus. Gothic, and…

      Ken: Yeah, for sure. Yeah, very interesting architecture — the art and architecture building in particular. Well, anyway, so I became more interested in photography. I wound up getting a job at a newspaper in the New York area, “Newsday.” Did two years there working in their editorial library in their — with their photo archive. But then I kind of decided that wasn’t really going anywhere fast enough, so I moved to Japan. <Wow.> And I had a three-part plan for going to Japan. I was gonna photograph myself — make a portfolio of my own work. And I thought that it might be interesting to get some teaching experience. So I taught English. And I was chasing a girl. So, that was the…

      Frank: And not in that order, right?

      Ken: That was the three-part plan, right. Photograph, teach, chase a girl. I wound up catching the girl so we’ve been married for — it’s gonna be 25 years in…

      Frank: Oh, congratulations.

      Ken: …a couple months.

      Frank: That’s so awesome.

      Ken: So, after that, I took that — the portfolio of work that I put together, two years in Japan, and applied to a fine arts program at the Rochester Institute of Technology, for a Master of Fine Arts degree program. But it was there that I discovered the World Wide Web. And so I put my plans to be a fine art photographer, or maybe a professor of photography — or you know, putting together the teaching experience with photography. I just set that aside, because I saw the web for the first time. It was probably 1994, and I thought it was the most amazing thing.

      I saw Mosaic and the professor, oddly enough, loaded up, you know, one of the few websites, comparatively, that was available then. Yahoo, when it was text only, right? And so to me, the interest was — I wanna make photos show up on this thing. Let me take my experience, my love of fine art and the liberal arts, and figure out how to make that come alive on the web. And I just wind up getting more and more into programming. I graduated — or, I left RIT without graduating with any degree. But by that time I’d learned enough to go get a job in a web development company and wound up making websites. And this startup, that startup, the next startup. I wound up at a company called Easel.

      Frank: Oh right, of course.

      Ken: I did Linux software development, making desktop Linux.

      Frank: Right. Every year is the year of desktop Linux. Every year for the last…

      Ken: Desktop Linux. We thought ’99 or 2000 was gonna be the year of desktop Linux, it turned out not to be, but…

      Frank: Not to be. But you worked on the Nautilus file browser.

      Ken: I actually worked on the portion of Nautilus that connected to these, sort of, proto-cloud services, right. And interesting…

      Frank: Dropbox before it’s time.

      Ken: Interestingly, for where I am here, Andreessen Horowitz — we hosted our cloud services at LoudCloud.

      Frank: Wow. Thank you very much for being a customer.

      Ken: And so, we went ahead with that project. But, of course, that company didn’t succeed. But of course, Easel had this long-standing connection through some of its principals. Andy Hertzfeld, Mike Boich, Bud Tribble…

      Frank: Yeah, the legends, right? Macromedia and…

      Ken: That got me an introduction to Apple. So I started [at] Apple in 2001, and started getting into making the web browser for Apple — was my first project.

      Early projects at Apple

      Frank: That’s fantastic. And why don’t we get into that story, because, as you tell in the book, you, sort of, started experimenting with the old Netscape codebase, right?

      Ken: Right.

      Frank: Called Mozilla, I guess, by then. But you ultimately didn’t go that way.

      Ken: Right. Well, you see, you know — it’s, sort of, interesting. And maybe we’ll get into this more as we talk. The way that Apple worked in this period, during the Steve Jobs era, is that he would set this vision. And so his vision was — Apple needs its own web browser. So at the time, when I joined in 2001, Mac OS 10 — the new version of the desktop operating system, replacing the old classic version of Mac OS that had been chipping on the computer since the ’80s, right. So came along with this Unix-based replacement. But that system didn’t have its own web browser. It was still part of the agreement that had been made a couple of years earlier with Microsoft, to provide Apple with a web browser, so Internet Explorer.

      Frank: When Bill invested…

      Ken: That’s right.

      Frank: …he brought Office to the Mac, and then IE became the default browser.

      Ken: Correct.

      Frank: People don’t remember this anymore.

      Ken: Correct. But that was the situation that Apple was in, is that this exciting new technology — the web — was something that wasn’t under its own control. And so, you know, the vision for Apple back then, and even still today, is that Apple wants to be in control of what it considers to be critical technology — critical to its user experience.

      Frank: And as all the operating system companies decided, right, the web browser was critical. It wasn’t an optional add-on component. Netscape and Microsoft famously got into a legal battle over this. So Apple arrived at the same insight. And then, interestingly, the two codebases that you consider to get Safari off the ground were Mozilla, right — the Netscape codebase — and then Conkeror, which was a Linux web browser. And they were both open source. And so talk to me about what it felt like at the time, to be looking at open source inside Apple — which is a famous, sort of, like, “We’ll build it all ourselves.”

      Ken: It was interesting that the executives — people like Avi Tevanian, who was the chief software VP at that time, and Stev — were willing to consider open source. But just to give you a brief summary of our full investigation, we considered writing a browser…

      Frank: From scratch.

      Ken: …from scratch. We also considered going out and licensing from a company like Opera. That was the company that…

      Frank: There were many licensing browsers back then.

      Ken: Right. And so, but we — but Don Melton and I, which was the two people — we joined on the same day in 2001 — to begin this browser investigation. And we looked at open source because we were a team of two people, and a web browser is a pretty complicated thing, right?

      Frank: Complicated. It’s harder than it looks.

      Ken: It’s harder than it looks. So we thought that if we could make a compelling case to use open source as a way to jump ahead in the effort, you know — stand on the shoulders of giants, right? You know, it would get us to a point where we would have something sooner. And that was really the goal. And, being open source, if we took the software from, say, another platform that — neither Mozilla nor Conkeror worked on the Mac. So we were gonna have this opportunity to bring this code from elsewhere and make it Apple’s own, and really make it look and feel like it was a native program to the Mac. So that was — and looking at that it really just came down to, Conkeror was one-tenth the size of Mozilla. And so as a two-person team — soon, thereafter, a three-person team — this just was the easiest way to get from where we were to where we wanted to be.

      Frank: Yeah, it makes sense. I mean, people don’t remember this about the early days of the browser, but when we shipped Netscape, we had to do it on 20 platforms. So every build was a, “All right, here’s the one for Arix, here’s the one for Digital UNIX, here’s the one for AIX, here’s the one for HP-UX. And here, by the way, is Windows 95, Windows 98, Windows NT. Like, it was such a cross-platform exercise, you know — the codebase, sort of, grew and grew.

      Ken: Sure. And so we only had to do that once, in that we took this Linux code and brought it over to the Mac. And, of course, it was a challenge for us. So I can only imagine what it would be to kind of keep all of these platforms going concurrently, as you’re trying to make improvements and add features and make things better.

      Frank: And so you ultimately decided on the Conkeror codebase as, sort of, your starting point. And then pretty early in the development process, you ended up building a stopwatch, the PLT. And so, maybe talk a little bit about — why did you decide to do that — and then ultimately, flashforward. Like, when Steve announced the browser, he would say, this is the fastest — like, it was one of the key features. And did you know at the time that you built the stopwatch that he was gonna do that or, like, did you get lucky or…

      Ken: So no we didn’t — it was not luck at all. Steve was very, very clear to us at a very early stage in our browser development process — was that — well, of course, he wanted to deliver the best experience out to customers, that was it. He wanted to put a smile on the user’s face, right? And so if you think about the challenge that we had, there was this existing browser on the platform…

      Frank: Right, Microsoft.

      Ken: …that people were familiar with, right? And so now we’re gonna come along and say, no, you had that other thing — here is this new browser that we want you to use. It’s Apple’s own browser. And, well, what is gonna convince people to make the change? And so Steve thought, well, we’re gonna need a compelling argument. And to be compelling, it needs to be simple. And so his idea — his vision — was look, we need to make this thing perform fast. Again, thinking back to the time that — well, the network wasn’t so fast. I mean, some people were getting, you know, maybe broadband at the office, but certainly, at home, you’re still doing dial-up.

      Frank: You’re definitely in dial-up.

      Ken: Right? And so anything we could do to speed up the browsing experience was something that would be attractive to people. People would notice. And so he said, “Browser team, you need to figure out how to make this browser fast.” And he told us this a year-plus ahead of time. So this PLT — the “page load test” is what PLT stands for — was this performance tool that we used during our daily development, so that every code check-in that we had, we would run our page load test to see that there were no speed regressions. We had this idea — that was really Don Milten’s idea, who was the manager of the team.

      He had this little bit of sneaky logic, where he said, “Okay, team. If we check-in code and it doesn’t make any speed regression, only two things can happen. Either the code will remain the same speed, or it’ll get faster, right?” And again, it’s just one of the simple things that just turns out to be this profound truth. Because as we would go over, you know, the weeks, the months, hundreds and hundreds and hundreds of check-ins, that’s what happened. Either the code either stayed the same, or it got faster. And over time, because there was the speed priority coming straight from Steve, we would look for ways to make it faster. And eventually, Safari, when it was released — it was three times faster than MS IE at loading web pages.

      Frank: That’s fantastic.

      Ken: And the point is, again this — Steve Jobs going out on stage, he has this reputation of being this great marketer, the reality distortion field. Anything that Steve says you’ll believe, just because he has this — through the sheer force of his personality. But this was more of a matter of just — of him just saying, “Well, we executed on this plan, we got a great result, and here it is.”

      Influence of Steve Jobs

      Frank: So, I love this idea that, sort of, Steve set this goal early on. Ship the fastest browser that you can ship, because when I launch it, like, that’s what I’m gonna talk about. And as I was thinking about, sort of, basically the software development process. You know, it’s rare for a CEO of a big company — and Apple was a big company back then — to be so intimately involved in the planning process. And, sort of, how important do you think that was to, sort of, your age of design?

      Ken: I think the way that Steve organized the company and built the teams — built the culture — was an essential part of how we did our work. And the way I like to describe it, is that Apple was this wonderful combination of top-down leadership and bottom-up contributions. So, Steve — the top-down part, I think is almost well-known. Steve was very, very clear. He could be almost, you know, domineering, right, in pushing his vision forward, right? So when you worked at Apple in software development, you knew what the vision was. That was always very, very clearly communicated. But it still was just a vision.

      Now, sometimes he would get specific, but most of the time, he just would tell us, “I want a great browser, and it’s gotta be fast.” And so with that as a brief, handed over to the engineering team, it was our job to figure out how to do it. And so then that’s where the bottom-up contribution comes from. He didn’t say, “I want you to make a performance test, and I want you to institute this policy where every check-in doesn’t allow any speed regressions.” No, we came up with that. Providing that bottom-up contribution that helped to realize the vision.

      And then one of these other things — and perhaps we’ll get into it a little more as we go, because it is such an important part of Apple’s culture — is that there would be demos. So, we would periodically — I remember quite clearly, there was a 0.1, there was a 0.2 demo — where we needed to demonstrate the strength and the potential of this open source idea — of the, sort of, the Conkeror source code that we had chosen. And of our porting plan and efforts before they would commit to going through to the project — to go from 0.2 to 1.0. So…

      Frank: It was Steve at the demo at that point?

      Ken: He would see the code yeah. Very, very often very, very…

      Frank: So that’s a little unusual. I compare that to, sort of, a typical Silicon Valley company where, like, you’re doing these demos frequently, right? And so in general, you, sort of, think of the CEO of a company this size not being involved in every single milestone, right? Because you’re Safari on Mac OS —Mac OS is one of the many products that Apple was shipping at the time. And so, like, it seems unusual that the CEO would be involved in this many demo points. And how important do you think that is to, sort of, the…

      Ken: See, I’m actually gonna dispute one of the things that you said, if I may — is that certainly during the Steve Jobs era — and I still think to today, here in 2019 — Apple didn’t ship a whole lot of products. Back then, Steve quite famously, when he, you know, re-established control over the company, he came up with that product matrix, right? Where we’re gonna have, you know, [a] consumer product, a pro product, a desktop product, and a portable product, right? And so we’ve got four products, and it’s the same operating system, right —Mac OS — and so it’s actually very, very few products.

      Now, interestingly, when I joined Apple in June of 2001, Mac OS 10 had come out. And so we had that two-part product matrix that we were still working in. And that was still four months before the announcement of the iPod, which was just that beginning of Apple expanding out from being — well, Apple Computer to being Apple Inc., right? You get into the more consumer-focused products that weren’t really thought of as being computers. But because — I mean, the point of going through all that, is that since there were so few products, Steve could keep tabs on what the software teams were doing. That there was this big initiative to make a web browser so he could his — keep tabs on it. He could find the time on his schedule to get updates on how the software was doing, and he did.

      Frank: So it was, sort of, a focus thing, right? By Steve saying, “Look, we’re not gonna have that many SKUs, we’re not gonna have that many products — like, then I can put all my eggs in one basket and watch that basket very carefully.

      Ken: You say the word — and it is one of the best words, perhaps the best word, to describe Steve’s approach — which is focus. Focus on what? Great products. I mean, there, in those three words — focus, great products — you can distill down Steve’s approach — his formula to just a couple concepts.

      Frank: So you ship Safari — awesome browser, fast, native, you get a lot of people to switch over. And then at that point in your career, after having been this individual contributor that, like, shipped this awesome product, you thought — like many people in your shoes — time to be an engineering manager. So, maybe talk a little bit about that story of, sort of, you know — how you thought about it, and then how you got the job, and then what the job was like when you got it. As your first engineering manager job?

      Early work with iPhone

      Ken: Well, you know, I always try to think about, well, what’s next? And I don’t really have a big career vision because — especially the tech world, it changes so fast, right? And so, it always seems like you come to the end of one thing, and then that’s the moment to really decide what the next thing should be. And as you say, engineering management seemed to be, like, this new domain that I didn’t have a lot of experience in. So I thought that this would be an interesting opportunity. And so I pushed for it, I asked for it.

      And it was actually Scott Forstall, the software executive — really instrumental in coming up with a lot of the, you know, interesting user interface work in the iPhone software project later, which I’m sure we’ll get to — but he was the one who was in my management chain who gave me this opportunity. And so I started working on the sync services software for the Mac, which at that time, was really still the software that would be up in the cloud, and would help two Macs sync with each other. I mean, we didn’t really have…

      Frank: There were no phones, no iPod.

      Ken: Right. Okay. So it’s like, you have a desktop computer in the office, you have a desktop computer at home — or maybe you have a portable and a desktop. And it was to get those systems exchanging some data — your contacts, your address book, things like that. And so, I thought this was, you know, an interesting challenge, and you know, people were gonna be getting more devices and things like that. But I found that very soon after I got into the job that I was miserable. That I hadn’t really reckoned, at that point in my career, with what management really is. It’s about people. I was still — certainly at that point in my career, still fascinated by the software itself.

      That’s what was [attractive] to me — about sync. It seemed like this distributed computing problem, and I was enamored of the technology, and you had client server, and you know, and all of this. And not really, again, thinking about how the right focus was to build a team, build a team culture — support the people so that they can do the technology. And again, at that point in my career, I wasn’t really ready for that, and I found myself within just a couple of months, I was miserable.

      Frank: It’s the lament of a lot of, sort of, first-time managers, which is — you think, on the other side, of course, “I want a manager job. It’s the way up, it’s the natural hierarchy.” And then you get there and your job is about shipping a team and not a product. And a lot of people go through that. I didn’t wanna ship a team, I wanna ship a product. So it sounds like that’s what you did — you, sort of, went back to being…

      Ken: Yeah. Well, I had, I’m almost ashamed to say, you know, it was like a mini-meltdown. I went to Scott Forstall and I say, “Hey, look, Scott. I don’t wanna do this. I led you astray, led myself astray. I quit. I offer to resign.” Because — and part of the thing is that it was a feeling of responsibility — that I had taken on a responsibility that now I did not want to fulfill. And I felt like, well, the only thing for me — there’s really just two choices. I could continue on being miserable about it, or I could just go and say, look, I’m done with this. You know, I submit my resignation. And so Scott was like, “Whoa, whoa, whoa, just a second…”

      Frank: Not quite yet.

      Ken: …stop right there. I wanna understand what’s going on there.” So I explained to him what I’ve just explained to you, about really wanting to still be in closer touch with the technology. And so he said, “Oh, okay, well, just go away.” He was not pleased with me. But…

      Frank: We got you the management job you asked for.

      Ken: You said that you wanted [it], right? And now you’re coming back, and now a couple of months later, saying that you want something else. What’s going on? So yeah, he wasn’t that happy but he had…

      Frank: And at that time, you had, sort of, started taking calls from Google recruiters, right?

      Ken: Yeah, I mean, because I thought that I was resigning. So I need to go get another job. So I actually did, and went to Google.

      Frank: Full interview cycle?

      Ken: I went and did the interview process at Google, and they offered me a job.

      Frank: So you were serious, you were ready to go.

      Ken: I was serious. I was serious. But turned it down — turned down, you know, that job, because Scott continued to engage with me. And he said, “Just kind of sit tight, maybe we’ve got something for you.” And a couple of days later, it was actually my direct manager at the time — said, “Come here.” And he took me into his office and he said, “We want you to work on this new project, sign this paper.” And I kind of thought there was just the barest little hint on the grapevine. So I just, like, reach out, I signed the paper. And he said, “Yeah, we’re making a cell phone, and you’re now on the team.”

      Frank: So that’s fascinating, right? So this is a great part of Apple that’s, sort of, very different than most Silicon Valley companies, which is — in most Silicon Valley companies, if you get assigned to another project, there’s not this level of secrecy. You’re not signing papers saying — so tell me a little bit about that. Like, what did they read you into at the time? It was Purple at the time, right, was the codename.

      Ken: The funny thing is that at Apple, I was already under this blanket non-disclosure agreement…

      Frank: You couldn’t say anything about it.

      Ken: …for all the — I mean, for the whole time that I worked there, I was under these document retention orders. I would get these periodic emails from the lawyer saying, “Do not destroy anything,” because of the work that I had done was then submitted in patents and, you know, perhaps there was gonna be patent litigation. So this is just the whole mindset, the whole culture of what Apple was. There was [a] secret, we were doing patentable — where we were trying to innovate. And we were, you know, interested in treating that work as a trade secret, something that was valuable, you know, to the company. And so…

      Frank: So, already super-secret culture.

      Ken: Already.

      Frank: And then you have to sign something, which is — I’m gonna introduce you to an even more secret culture…

      Ken: Even more.

      Frank: …inside Apple. It’s kind of like, you know, when you do the logic classes, like — infinite sets can be larger than other infinite sets.

      Ken: That’s right.

      Frank: Now you’re into the larger area — like, seriously? Like, what?

      Ken: Now you’re in a bigger, deeper, darker…

      Frank: …there’s more secrecy?

      Ken: … infinity. That’s right, it is a bottomless well, truly. So, yeah — I had to sign this additional NDA. And yeah, I got introduced to this project, it was called Purple.

      Frank: Purple.

      Ken: The code name for iPhone, and it was in development. And my job was to join the software effort, which at that point was maybe six or eight people, to do…

      Frank: That’s a tiny team.

      Ken: Tiny little team, to do what I like to term the high-level software. The plan was that we were gonna take as much of the Mac as possible and bring it over and squeeze it into one of these — you know, tiny, little, you know, smartphone form factor. So we’re gonna take the operating system kernel, and some of the low-level libraries — you know, the networking stack, things like this — the graphics stack. But above the level of core graphics, which was, you know, the low-level graphics library — above that, it was then — I was invited onto the team that was gonna invent the touchscreen OS.

      So, we weren’t gonna take any of — naturally, the mouse tracking or handling, or anything of app kit, which was the, you know, the user interface level software for the Mac. We were gonna make that from scratch for the phone. So what became UIKit, for people who know about, you know, the technology for what became —  you know, iPhone software iOS — that was our job. And so we started with a clean slate, and that slate was pretty well clean when I joined. Again, just about six or eight people on that effort at the time.

      The iPhone design team

      Frank: Yeah. So they tap you on the shoulder. You’re on the Purple team. It’s like six to eight people. So tell me about the people on the team. Like, what are the roles — are there product managers, are there UX designers?

      Ken: Right. So when I say six or eight people, that was software engineers. There was also this other team of designers, which in Apple we call the human interface team — the HI team, human interface. And that was the team of designers — they would do graphic design, animation design — but they would also do concepts. They would provide the thinking behind what is going to be the experience of the person that is gonna be using this product that we make? And so there was the small team, half dozen software engineers, and HI designers, and then executives — managers. So there was a fellow named Henri, who was leading the software engineering team. There was a fellow named Greg Christie, who was the day-to-day manager…

      Frank: HI.

      Ken: …of the HI team. They both reported to Scott Forstall, who was the executive who reported to Steve. And that’s it.

      Frank: And that was it.

      Ken: That was the team. Now, eventually, we wound up adding, over time, more people. We probably never had more than 20 software engineers, and maybe 10 designers. Those two managers, and the executive, and Steve, and that was it. And so…

      Frank: Super interesting, no product managers

      Ken: There were no product managers.

      Frank: No product managers, no QA engineers, like, until later.

      Ken: Until later.

      Frank: So the core that, sort of, got the whole product going is software engineers, human interface designers, and executives. And that’s it.

      Ken: Yeah, then we added, then, a program manager. So there were maybe, like, two people in just managing the schedule, tracking risk — you know, looking at the bugs. A couple of QA people joined. You know, at Apple — you know, certainly from my standpoint, you know, I consider them engineers. You know, they’re the QA engineers, right. But still, that still is all-encompassed in the numbers that I gave you. And, in a way — I say there were no product managers, but I would say that we had one product manager. There’s two ways that I could say it. We either had one product manager, Steve.

      Frank: Yes, the ultimate decider.

      Ken: Right? Or that we all were. We all were, it was all…

      Frank: So that’s really interesting.

      Ken: …our responsibility to make sure that the product was going to be great for people. We all shared commonly in that responsibility.

      Frank: So that’s really interesting, because you, sort of, distribute the responsibility. Now it’s everybody’s responsibility. But, you know, a lot of companies would think, ooh — I’ve gotta have a throat to choke. I’ve gotta have, like, the one person. But of course, at Apple…

      Ken: So we did.

      Frank: Right. The one person was Steve.

      Ken: And then another way, when you get down to the level of features, we had this notion at Apple of directly responsible individuals…

      Frank: Oh yeah. Let’s talk about this.

      Ken: So we had DRIs, right? And so when I started working — when I was invited to join the Purple effort — because of my experience on the web browser, I started working on crunching down Safari. Optimizing Safari, so that it could fit on a smartphone operating system and form factor. But then, after a couple of months, we had a bit of an impasse with the software keyboard. And we had what was really quite unusual — really unique in my experience at Apple — is that this was judged to be — the development of the software keyboard was judged to be sufficiently high risk, and that the risk was not being matched by commensurate progress, right? I mean, the whole thing was high risk, right —we’re gonna make it a whole new touchscreen operating system, right? So the whole thing was high risk.

      But the thing is that we were making good incremental progress on most of those areas — touchscreen, and the UI kit, and Safari, and messages, and calendar, and you know, all of these — you know, the phone app. But the touchscreen keyboard was lagging behind all of these other projects. And so, one day — really, again, unique in my experience — Henri, who was the software engineering manager, called all of the engineers out of our offices into the hallway, and we had a group meeting. Again, about two dozen people, probably even less than that. And said, “Okay, you all stop. Stop what you’re doing. Stop working on your calendar, phone app —you know, the user interface level software. Everything, stop. Starting from now, you’re all keyboard engineers.”

      Frank: Wow, that is crazy. Like, the entire team…

      Ken: Entire team, stop.

      Frank: …everybody is a keyboard engineer.

      Ken: Because the idea was that if we don’t crack this problem, we might not have a product.

      Frank: Yeah. So I think we need to take people back to that era, right? Because this seems super counterintuitive, that you’d put all 20 people on one project. And so take us back in time. So, the most popular phone at the time was the CrackBerry — the RIM BlackBerry, and it has a physical keyboard.

      Ken: Has a physical keyboard. And so, this was in the fall of 2005. And again, to just give the time perspective — Steve stood up on stage and announced the iPhone in January of 2007. So again, this is a really, really compressed timescale. So, just a little bit more than — you know, it’s less than a year-and-a-half out from the day where we were trying to hit that target.

      Frank: Eighteen months, not a lot of time.

      Ken: And we still had really nothing to show for this effort to give a solution for our phone which would compete with the BlackBerry, right? And of course, the BlackBerry had this wonderful keyboard, the hardware keyboard, the little plastic keys, click, click, click, click — the little chiclet keys. And again, you said the word CrackBerry — people loved…

      Frank: Loved those things.

      Ken: …loved the products, it’s a great product, right? But we were gonna provide this different vision for what a smartphone would be. Is that it was gonna be this — that there wasn’t going to be enough room for a plastic keyboard with the keys fixed. We were gonna give more of the front of the display over to a screen, to software. And so the keyboard…

      Frank: And that was…

      Ken: …had to be in software.

      Frank: And the idea of an all — for, sort of, software-based keyboard was one of the design things that came from Steve early like it was…

      Ken: Yes. 

      Frank: ….just like, look, this is not negotiable. I’m not shipping a physical keyboard.

      Ken: That’s right. His idea was that we need a keyboard some of the time, but we certainly don’t need it all of the time. And so the idea of the keyboard being in software is that it could get out of the way, it could go off the screen. Which would then make the rest of that screen real estate available for a customized user interface that was great — that was optimized for either the phone app, or if it’s the calendar, you can see more of your appointments, or see more of a month view for the calendar. So it was absolutely essential that the keyboard could get out of the way when you weren’t using it, so that the device could be opened up for these other, better, richer experiences in the apps that we were gonna be shipping.

      Frank: And what problems were you running into at the time? Like, were people missing keys, were the keys not big enough? Like what caused the…?

      Ken: You know, again, I mean — in some ways, it’s hard to think back, given how history has played out, right? That we have our phones now, and — you know, maybe you’ve got — you know, I’ve got my phone here today, and I’m two-thumb typing, and I’m hardly even looking, or whatever. Back when we were working at this early stage, and we were all new to interacting with touchscreens, we found that we had this real sense of apprehension. Apprehension, whenever we were gonna touch a target on the screen that was smaller than our fingertip, right? That was actually a really interesting threshold. A constraint that we were dealing with when we were designing the user interface, is that if the target that you were going for was larger than your finger, you could target because you could maybe move your hand a little bit out of the way, and you could see what you were going for. If the target was smaller than your fingertip, like, did I get it? I don’t know, right? And so we started…

      Frank: You don’t have tactile feedback.

      Ken: We didn’t have the tactile feedback of that BlackBerry, right? You could feel the edges of the keys with your fingers. And of course, with the touchscreen, it was just this sheet of glass. And so that’s the challenge with the keyboard, is that you needed enough keys to have a typing experience, right? But in order to give the number of keys necessary, the keys needed to be smaller than your fingertips. So what do you do? And so, it turns out that, you know — through investigation and lots of demos and lots of sleepless nights, right, that the way to close that gap was to give software assistance.

      Frank: And so Henri waved the magic wand. Everybody now is a keyboard engineer, everybody needs to figure out how we’re going to make a reliable keyboard that’s delightful. And so what happened from that point? Was it like a series of demos, where people were just…?

      Ken: Yeah. We did this series of demos. See, again, going back to the way that it was in that hallway — and it was just one hallway, since it was so few people, sort of, 20-ish people. And we all had our individual offices at the time. This was not [an] open-plan office, right? Everybody had their office. Mine, when I was working and thinking, I had my door closed, right? But then, okay, so I would be in my office with my door closed, and I would come up with a demo — an idea, right? — that could be represented in a demo. Then I open the door, and I go to see who else’s door is open, and say, “Here, try this,” right? And so we would have this culture, we were all demoing to ourselves all the time.

      And when we were set off on this thing, you’re all keyboard engineers — we all just went in our own directions. Some of us, you know, had already well-established — you know, collegial relationships where I would collaborate a lot with you. And some other people, you know, they had — maybe they work by themselves. Some people had a good relationship with one of the HI designers, or whatever. So we just cobbled together our own little teams, our own little efforts, and started making demos.

      And again, trying to combat this problem of the keys being too small. So one idea that we experimented with was making larger keys with multiple letters on the keys. I started experimenting with software assistance — maybe there could be a dictionary on the phone that the software could consult to provide suggestions that maybe, you know — much like we have today, that there’s this bar on top of the keyboard that is updating as you’re typing keys, giving you some notion of what the software thinks you’re trying to do.

      Frank: Autocorrect. The author of autocorrect, which is now not only super useful on the phone, but probably my favorite comedy genre. You know, go watch the Facebook videos. Autocorrect comedies, they’re fantastic.

      Ken: Well, sorry about that. So eventually, you know — the breakthrough, if you will, that made it possible for software keyboards to really work, you know, in a shippable product was software assistance — to the extent that the software may change the letters that you type. That it’ll change it to what it thinks, rather than what you did. And actually, this phrase is really, really important. I think really, really — one of the important organizing concepts for so much that we did to make the touchscreen operating system work is because you didn’t get this tactile feedback, because you couldn’t feel the edges of either keyboard keys, or any button, or anything in the user interface — is that the software had to be there working behind the scenes, to give you what you meant, maybe differently than what you did.

      Frank: And how did you come up with this idea? Because this is a classic “thinking outside of the box” idea, right? Like if you were gonna try to solve this problem, I bet you saw a lot of variations of, sort of, key sizes — and you know, that type of thing. But like consulting the dictionary putting up suggestive words, like where did the idea come from?

      Ken: It’s just this iterative process. It just takes, you know, a long, long time. You start with ideas, maybe somebody else does a demo — does an idea, and you had your idea, and you think, “Oh, maybe if I can combine those two ideas and make a demo of the best of everything that I see.” And it was just this collaborative soup of ideas all swirling around, and you just take the — you know, all of us were — there was a sense of friendly competition, and it was both of those.

      We all wanted to do the best. We all wanted to be the one. I mean, I think we all had a sense of maybe — a sense of ego, that we wanted to be the one to crack this hard problem that we were given. But it was all very friendly, in the end, that if your idea wound up winning — proving useful — yeah, you got a little bit of, sort of, geek cred for that on the hallway. Everybody knew who it was that came up with the idea.

      Secrecy at Apple

      Frank: I wanna talk to you a little bit about this, sort of, secrecy, right? You got led into the Holy of Holies — it’s more secret than, sort of, other parts of Apple. And at one point, you decided — as you were refining the autocorrect algorithm — that there were actually experts outside of the Purple team that might be able to help. But, of course, they hadn’t been disclosed. And so, like, what was that like to try to go get their help, and was it offered?

      Ken: It was tough — it required getting approval. It’s like, well, I’m gonna go and talk to these people, but there was no process really, at that point, to get them disclosed. I mean, really, you know, at a certain point, Steve was still personally approving every person that was submitted to get disclosed on the project. But I did get permission to talk to them. So as long as I told them, I can’t tell you why I want to know how, say, the Japanese input method works. You know, the way the Japanese works is that there is this input method — that there is a sophisticated way to take the keys that a user types and turn it into the Japanese language. A text that actually reads as Japanese.

      And so that — and I just won’t get into the details of that. But it seemed like it was similar, in a way, I mean, at least in the thought process. Is that we have this real software whirring away in the background, other than — you know, different than say, just like a desktop keyboard, where if you type the A, you get an A, right? And so I went and talked to them. But you know, in the end, it was just more of conceptual help than really, you know, anything concrete that I could put into the software. It just turns out, really, that the problem that I was trying to solve, which is really input correction — that you weren’t sure what key you hit — was a class of problem that was different enough that it really required different solutions.

      Frank: Looking back at it now, which is, sort of, the extreme secrecy — you couldn’t really describe the problem, right? And so as a result, you got some conceptual help, but not, sort of, concrete design help. Would you think of the, sort of, tiers of secrecy inside Apple as a feature or a bug, or somewhere in between?

      Ken: Yes. You know, the thing is, I think there is a really underestimated power in keeping your team small. The cohesion — the small unit cohesion that you have, where simple things — like, we’re gonna have a meeting. Who do we invite? Everybody. We’re gonna have a team meeting, right, where we’re gonna talk about important milestones. We’re gonna call everybody out of their office. Henri could say, “Hey, everybody, come out of your offices, please.” And within 30 seconds, everybody was…

      Frank: Everybody is there.

      Ken: …standing there, right? So, you know, you get these — there are advantages to keeping things really, really small. And of course, then there is the disadvantage that when you are trying to tackle difficult problems, you may not have all of the talent that you need. And you may not have a sufficient amount of diversity, right? That all the — you know, especially, you know, a company like Apple is trying to make products for everybody. Well, how do you design for everybody, if the design team isn’t a microcosm of everybody? And so, there are these really profound challenges, right? You know, back in these times, we did the best that we could within the constraints, you know — and we tried to then really tap into the benefits that the smallness and the secrecy gave us as well.

      Frank: Another funny thing that I learned reading your book is the secrecy was so extreme that, like, you didn’t even know what the product was gonna be named. And so, like, the word iPhone wasn’t even in the dictionary…

      Ken: That’s right.

      Frank: …like after Steve launched.

      Ken: That’s absolutely true. So there was — we were all heading toward this announcement for the iPhone in January of 2007. So if you remember how Steve introduced the product, he said — you know, he gave his very dramatic introduction, you know. So he said something to the effect of, well, we’ve got you know, a groundbreaking product — you know, and you’re privileged to be involved in, you know, a product like this maybe once in your career. But Steve, he had been involved with, you know, the Mac, and then the iPod. And he said, “We’re gonna have three new products of this class today.” And I’m saying like, wait — there were two other secret projects that I didn’t know about? I mean, truly, for a moment I didn’t get — and it’s like, oh, no, no, no, it’s just how he’s gonna tell the story.

      Frank: It’s my product he’s talking about.

      Ken: That’s right it’s gonna be…

      Frank: That’s awesome.

      Ken: …you know, the phone, and it’s, you know, gonna be you know, the touchscreen music player. And then, you know, the internet communicator. And no, this is actually all just one product, and we call it iPhone. And when he said that, that’s when I knew that I was gonna have to go back the next day and add iPhone to the autocorrection dictionary.

      Frank: That’s awesome that he fooled you too, because he fooled me, like, I — and like you were working on it, so I don’t feel quite as bad.

      Ken: Again, the secrecy. You know, I have to admit that it was just a moment where it’s just like, wait, wait a second — is there something that I don’t know? No, it can’t be. But yeah, that was just the culture and the times and the way Steve liked to run things.

      Frank: Now, a feature we all take for granted now actually didn’t appear in iOS until several releases later, and that’s copy and paste. So I wonder at the time, did you guys talk about that? And did you make an explicit decision to, sort of, like, yep — let’s ship without copy and paste? And was that contentious? Because on the surface it would seem like that’s contentious.

      Ken: Yes, it was. But one of the other things that we were really expert at, to bring back the word that we had talked about earlier — was focus. In that, we were very, very good — really very, very early in the development process — to say what was in, and what was out.

      Frank: Physical keyboard out, super early.

      Ken: That’s right, very, very early. And that it was clear that this was — that getting the text entry system working at all, was going to be one of the real challenges. I got used to being in the team meetings where Henri, team engineering meetings — again, everybody is in the room. So we got 20 people in the room and Henri is, you know, up at the front of the room, and he’s got you know, a Keynote slide deck. And he’s saying, okay, big challenges — well, keyboard, of course, you know, and then whatever other challenge there may have been.

      And those challenges came and went, but [the] keyboard was just a constant throughout the whole, you know, 18-month development cycle. And so, we knew that we wanted copy-paste, but we knew that there was simply not gonna be time for it. So we didn’t spend any real development effort on it. The one thing that I did implement for the first iPhone was the loop. So you press and hold, and it would give that little magnifying…

      Frank: The magnifier.

      Ken: …glass above your finger that would show. And the whole idea of that is that we wanted your finger to be right where the insertion point, you know, the little cursor would move. And so then we needed to show you what — and so this was an idea that I came up with. But then there was no time to capitalize and expand on that to do cut copy, paste. And it even got delayed an extra year, because in the second year, after we did the initial release of the iPhone — and then we had that six-month delay before we did the first customer shipments — and then that whole next year was taken up by making third-party APIs.

      Frank: So, two releases before you run copy and paste.

      Ken: That’s right.

      Frank: And so I wanna get right into this because, sort of — look, Apple was famous for having exquisite taste around the design trade-offs. And a feature like copy and phase kind of feels like wait — you’re arguing against copy and paste? Like, that’s not a great user experience? And so, like, how did the argument evolve? And, sort of, the big setup is, there’s taste — taste-making, making hard decisions like this. And then there’s, sort of, another style of decision making which, sort of, Google made super popular — which is just relentlessly A/B testing everything, right? And so like, maybe the way Google would have come at this challenge is all right — let’s give people tasks. This one has copy and paste, this one doesn’t have copy and paste, let’s A/B test it. But Apple made, sort of, like what I would argue is a pretty courageous call, right — that seems to fly against the user intuition. To exclude it.

      Ken: Well, it was simply a matter of setting the constraints and keeping them. You know, again, you know, maybe if we had doubled the size of the team, we could have gotten some other things done, but maybe not to the same level of quality. And again, once you start adding people, other things begin to break down, right? You can’t invite everybody to the team meetings, you can’t find a conference room big enough, right?

      Frank: Right. And now, there’s 40 people who can break the build.

      Ken: That’s right. I mean, [it’s] how you start to have problems like this. And so we just decided that, well, you know — it’s, like, a Steve way of maybe communicating this was — look, this is the greatest product ever. The touchscreen iPod, it’s the greatest iPod that we’ve ever shipped. It’s got all these great features. It’s a phone, it’s got web browsing that you can take anywhere with you now. And there’s no copy-paste — well, who cares? We’ll get to it, right. I mean, in the meantime, you’ve got this, you know, the most amazing product that we’ve ever made. And so, that was — and Steve just was — you know, in his mind, he believed that the things that we did do, were good enough to counterbalance for the things that we couldn’t do.

      Demoing for Steve Jobs

      Frank: So that’s great — great segue to, sort of, the next segment. I’d love to, sort of, take us into what it was like to demo for Steve. Like, what was the room like, who’s in there? Like, what’s the emotion of it? Everybody wants to know this, right?

      Ken: It’s pretty…

      Frank: It’s probably the scariest room in Silicon Valley.

      Ken: It was pretty scary. Steve could be intimidating, there is absolutely no doubt about it. But you know, to get back to this point I mentioned before of the top-down and the bottom-up — as I mentioned, except for this very brief interlude where I was a manager, throughout my whole Apple career — over 15 years, almost 16 years — I was an individual contributor. And yet I got the opportunity to demo to Steve some of the latest work that I did at various points in my career. Because he wanted to see from the person who did the work.

      And because when he would ask questions, well — go and ask the expert, right? Go ask the person who is the DRI, right, the directly responsible individual. The person who is — at least according to plan — the person who when they lose sleep, they’re losing sleep over that thing, that they’re gonna be demoing to me. So that’s what he wanted to do. And these demos were very, very small affairs. Now, interestingly, the demo room for Steve — the software demo room — was this really shabby little room.

      Frank: That’s not what you would expect for Steve Jobs…

      Ken: You wouldn’t think you think…

      Frank: …coming out of performance, right?

      Ken: …would be this pristine room …

      Frank: Exactly, [a] beautiful blonde woman.

      Ken: Like you know, air filter — the air is clean or, you know — like, the scent of redwoods, or something like that piped in. No, no, it was this shabby little room with this mangy old couch, and just standard-issue office furniture. And that’s what there was. I don’t know why he didn’t want better. But the only reason that I can say is that, again, it was a matter of focus. He was focused on looking at the software and not worried about the decor.

      Frank: All right, so take us in the room. It’s a mangy couch. Who’s in the room? Let’s do the version where you’re trading off, sort of, the keyboard with the big keys, or the keyboard with the little keys.

      Ken: Okay, so skipping ahead a couple of years after the original iPhone, when we were then doing the original iPad. So this is now 2009, as I recall — so a couple of years later. And so this is actually an original iPad right here…

      Frank: Lovely.

      Ken: …and it’s actually a good one, which is autographed by Steve Jobs. So this was the iPad that I got at the end of the iPad development process. But back at the beginning of the iPad process, you know — I would have a prototype that looked pretty much like this. And so we were thinking of, well — what’s the typing experience gonna be like? And so here’s an original iPhone, an original iPad — well, we’ve obviously got a bigger screen.

      Frank: A lot of pixels now.

      Ken: Right. So now what are we gonna do to make great use of these additional pixels that we have? And one thing that I also noticed was, if you turn the iPad to landscape, that screen distance is actually just about the same as the distance between the Q key and the P key on a laptop keyboard. So I was thinking, hey, like — wait a minute. We could maybe fit a full-size — something that is a full-size keyboard — on a landscape iPad.

      Now, it turns out that right around at the same time, one of the HI designers — one of my favorite HI designers that I really loved working with, and who I’d also collaborated with on the iPhone keyboard, Bas Ording. He was starting to think about iPad keyboards, as well. And so, he had come up with this demo where he had all of these variations, all of these ideas. And so, he gave me a demo where he went through — he showed me, you know, 10, 20 different ideas. But one of them made — really struck me, which was — he had a design that showed pretty much just a shrunk-down laptop keyboard to fit in this space. And so, what that meant is that I had two ideas — it’s that maybe I could use this larger screen real estate to make a version of the keyboard that had big keys, that was almost the same size as a laptop keyboard, but then one that also gave you, like, the number row and all of the punctuation keys exactly where you would expect to find them on a laptop keyboard.

      And so I figured, well, you know, and I first started talking with Bas, and we came up with this demo where we would have a special key we called the zoom key — that would take you from this keyboard that had the small keys — that would zoom up to the larger keys, and then back down to the smaller keys — as a kind of a complement to the globe key that changes the keyboard language. So we would have this other key — this kind of complementary key — that would change the keyboard layout. We thought this was a great idea to, you know — and again, the idea of, what are we gonna do with this larger screen real estate for the iPad, right? A software key.

      Frank: The idea was, give the user choice.

      Ken: Give the user choice.

      Frank: I have these pixels.

      Ken: Give the user choice.

      Frank: I have the real estate.

      Ken: Use these new pixels that are available on this new platform, this new form factor, and have that be the pitch that we make to people. And so before, of course, you can make the pitch to people you need to make the pitch to Steve.

      Frank: To the man.

      Ken: That’s right. And so, I got to demo this for Steve. And so the way that this worked is that there was a very small team that was, like, the chief demo review team. The small group of people that Steve wanted around him as he was reviewing demos. And this was Scott Forestall, Greg Christie, Henri — people that I’ve mentioned — so, you know, the chief managers for iOS. And then a couple of HI designers, like Bas Ording, the fellow that I collaborated with on this keyboard, was, you know, almost always in this meeting. Another fellow, Steve Lemay — another HI designer — was often in the meeting. But as I recall, he wasn’t in this particular one where I was demoing the keyboard.

      Frank: So half a dozen people-ish.

      Ken: Half a dozen people in the room, yeah. And so then what would happen is that people like me who had individual demos — so it’s, like, there were circles inside of circles. So I was in the circle of people who could demo to Steve. But then there was this circle inside of that who would stay for all the demos. And so my role would be — or my — you know, how I would figure is, I would go in, give my demo, and then leave. And so, you know, think of that — beforehand is that, you know, I’m sitting there with my iPhone you know, down the hallway, waiting for Henri to text me.

      Frank: Waiting for my turn.

      Ken: That’s right. You know, and so he sends me a text, “Go stand outside the door,” and then, you know, the door is gonna open, I’m gonna get invited in. So I get the text, I go stand outside the door. And you know, I’m waiting, and I’m waiting, and I’m waiting, and it just seemed like, well — he just texted me. Why did he text me? And so then the door opens, I get invited, and I figure I’m on. I’m gonna go do this iPad keyboard demo. And I come around the corner and turn into the room, and Steve is over there and he’s like this. He’s like…

      Frank: He’s on the phone.

      Ken: He’s on the phone staring at the ceiling, like, you know, going back and forth in his office chair. And I’m like, gulp — I was like, what do I do? Like, now I’m eavesdropping on Steve on his phone call, right? And so you know, it’s pretty uncomfortable. And I think — I actually do think that he was talking to Bob Iger, the head of Disney.

      Frank: Disney.

      Ken: Disney, right. And so he’s like, “Yeah, Bob, yeah, that sounds great. Yeah, I’ll call you next week. Yeah, great talking to you.” So then you know, he hangs up. And then he does this thing, he takes his iPhone — he puts, you know, his phone back to his pocket. And then he does this, right? It’s like, I don’t know if you know — it’s like the Eye of Sauron, right? The “Lord of the Rings,” right? You know, the great eye turns to focus on you, and that’s what it feels like.

      And so it’s very, very interesting, then, how the demos go from that point — in that, he didn’t want a lot of words. He didn’t want a lot of, you know, used car salesman pitches, right? All he really wanted to know was what was next. And so what happened is, he hung up the phone, he turns towards me. And then Scott Forstall was the one who then stepped up. He goes — and the iPad was already in the room. And so he goes and wakes it up and brings my demo up. And says, “Steve, we’re gonna be looking at iPad keyboard options now.” Ken, he did work on the iPhone keyboard, and now he’s got ideas for the iPad keyboard. So Ken.” So I said, “Yes, Steve, go and look at the demo it’s on the screen now. Try the zoom button.”

      Frank: And that’s it.

      Ken: That’s it. That was the intro. And so then Steve goes — he, you know, slides his office chair over, and he starts, like, looking at the iPad screen. And what was up was one of the two keyboards — let’s say it was the big key keyboard, the one that was more, like, suitable for touch typing. And he’s looking at it, he took a long time to look at it. It’s like, he even did this little thing where he was like, turning his head to see what it looked like, like in his peripheral vision. It’s like he’s just — it’s just incredible to see — what does Steve do when he evaluates product? Okay, so this is what — and it’s what he did. And he hadn’t even touched it yet, he was just looking at it.

      Frank: And this is going on for a long time.

      Ken: You know, it seems — it’s like one of those things where it was probably maybe 20 or 30 seconds that felt like 20 minutes, right? But he took a long time to study, and then, eventually, he goes out and touches the zoom button. And this zoom button to change between the two keyboards — in this case, shrinking the keys down to be the more laptop-like keyboard layout. The animation that Bas Ording had designed was one of the most beautiful things I’d ever seen. I mean, it really looked like they were — like the keys were just, like, morphing. It was absolutely beautiful.

      But Steve just was, like, no reaction. He does the zoom. And then he does this study again, he’s like looking at all the keys, looking at how the screen changed. Then he does the zoom again, and it goes back to the state that it was in the beginning. And then he studied a little bit more, and tapped [the] zoom button again, to see that it’s like, okay — there are just two states that we’re going here between, right. We’ve got two keyboards, I see the animation go between one, then the other, back to the first one. He satisfies himself that he’s seen what there is to see. And so then he turns to me and he says, “We only need one of these things, right?”

      Frank: You’re like, I’m on the hot seat.

      Ken: And I’m like, “I guess so.” And then he says — I mean, this is again the interesting part. He asked me, which one do you think we should use? He asks me. He doesn’t ask, you know, Scott Forstall — who is, you know, he knows much better. He doesn’t ask, you know, any of the other people in the room. He asks me, the individual contributor. You know, he’s coming in…

      Frank: You’re the DRI.

      Ken: But I’m the DRI you see, that’s the thing. He wanted the answer from me. Now, the thing was, I had to give an answer. You know, if I didn’t give a good answer, maybe I would never be invited back again.

      Frank: Not the DRI anymore, with that answer.

      Ken: And I had no idea that this was what he was going to ask. But in that moment, I came up with an answer, because I thought about my experience with these two keyboards. And I thought that, you know, the one with the bigger keys, I found more comfortable. I was getting to be, you know — that maybe with, you know, like four or five fingers, that I could touch type. And auto-correction was helping. So that’s what I said to Steve, I said, “Well, I like the bigger one, you know — the auto-correction is kind of helping, and I’m starting to get a feel for touch typing.” And he says, “Okay, we’ll go with that one.” Demo over. And you know, the interesting thing is that, then, that’s the keyboard that shipped on the product — with the slight modification of taking away the zoom button, which was now no longer needed, right?

      And so Steve had this amazing ability to simplify, and to rely on his people to have a good enough idea about what they were doing. And to be involved enough in the work that even when you get asked difficult questions, you know, about it, that you’ve been thinking about it. You have this background of just context, of having been thinking about the problem for weeks and weeks. That experience was then something he was interested in tapping into to provide a way forward for the product.

      Frank: What was going through your head when you were just watching him, sort of, head tilt in silence? Were you like, tempted to, like, explain things? Were you like…

      Ken: Yeah, well, you just know that that’s not…

      Frank: That you’re not supposed to do that.

      Ken: That you’re not supposed to do that. I would imagine that if [I] had done so, he would have been in no uncertain terms — he’s like, “Let me look at the thing.” Because now it’s like, what was he doing? He was, in my view — I don’t know what’s going on inside his head. But just having seen him do that — having at least, you know, enough experience with him and his approach to evaluating work — is that he was putting himself in the position of a customer. He was envisioning himself being in an Apple Store, as a customer, walking up to a table, seeing this new iPad thing for the first time. What’s gonna be my impression of it? So he pictured himself as customer number one.

      And so you know, I don’t want anybody — I don’t want the engineer, the engineers aren’t gonna be there to be whispering in the ear of the person in the Apple Store. Sure, they can maybe get the help of, you know, one of the nice people, you know, working in the Apple Store. But gosh, wouldn’t it be better if I can figure this thing out for myself, and decide for myself? And see the evidence of the care that the engineers and designers had put into the work. I can decide for myself — yeah, this is a thing I wanna take home with me, right?

      Steve Jobs’ lasting influence

      Frank: So, obviously, if you have a leader like Steve, that’s that into being able to emulate the user — who has great taste — like, you wanna make this person benevolent design dictator for life, right? Now, the downside of that, you know, Silicon Valley is getting a lot of criticism for these, sort of, super charismatic, “reality distortion field generating” CEOs. Where, like, you might not agree with them, right — and, you know, in the, sort of, ultimate downside case, there’s, sort of, just too much hero-worship of CEOs. Like, do you think that ever became part of the Apple culture, right? Sort of, the blind obedience to the fearless leader?

      Ken: Yeah, I think Steve’s reputation and his success causes people to draw the wrong conclusions, to take away the wrong lessons. I think that if you go back and look on YouTube of old videos with Steve — maybe, you know, on stage with Walt Mossberg and Kara Swisher at their, you know, AllThingsD Conference. Or, I just had a reason to go back and look at the Antennagate….

      Frank: Oh, I forgot about that.

      Ken: …because — and the reason I did this is because, you know, it’s current now that there was a bug in group FaceTime. And Apple issued an apology saying, “We’re sorry that we had this problem, and we’re gonna be fixing it,” whatever. And so I wanted to go back and see well, what did Steve say about Antennagate — you know, which was the issue with the iPhone 4, where you’re holding it wrong, and the signal strength would go down. And I wanted to see what he said. And it’s really interesting — this is on YouTube, you can go and look at it. And Steve held a little press event. And you know, he was just very, very clear — very, very upfront saying, “Our goal is to make our customers happy.”

      And so, that’s the kind of lesson that people should be taking away. It’s not that he was domineering. Not that he was this, you know, absolute monarch, you know —  21st-century absolute monarch now in a company rather than a government. Or, you know, all of that, you know — that he had this, yeah, reality distortion field personality. It’s that he had this focus on doing great work and making customers happy. That’s really what he cared about.

      Frank: And then, sort of, how did the organization morph itself to, sort of, reflect that you had this, you know, great tastemaker — who wanted to make these decisions that are, sort of — very granular level in the design? So there was an example where you were designing an animation. I think it’s, sort of, the scrunched zooming demo. And you got to the point where, like, Steve and Scott Forstall actually disagreed. So maybe tell us a little bit about that and…

      Ken: This was for iOS 5. So, this was, you know, maybe the second version — second or third version of iPad software. And we wanted to come up with multitasking gestures, is what we called them, so that you would have some way of interacting with your whole hand on the screen. Well, obviously, from the beginning — even though multi-touch was something that shipped even in the first Apple product, there was no way that you could have sophisticated gestures — multi-finger gestures on a screen that size. But with the iPad, we thought that you could.

      And so we had this idea of, well, what if you’ve got the home button that way, you still maybe want some gestures to interact with the device to control going between app to app. So I came up with this idea of using this five-finger gesture, like you take a sheet of paper and crumple it up and throw it away, to go from an app back to the home screen. There was then this other interaction where you would swipe side to side, to just go between one app directly to some other app, right? So you know, you launch Mail, and then you launch Safari — well, then I can just swipe to go from Safari back to mail, right? So that the system would keep track of the history of apps that you launched.

      So now, here’s the part that Scott didn’t like. So let’s say you start up your iPad from nothing. You know, you take it out of the box and you bring it home. And yeah, you launch Mail and you launch Safari, you’ve only ever launched two apps. So you swipe to go from Safari back to mail. Well, what happens if you continue swiping in that direction, right? There’s no other apps.

      Frank: End of list.

      Ken: End of list. And so what I came up with was this, sort of, morphing, stretching, rubbery distortion of the app to show you that you were at the end of the list. And it would kind of do this bloop, bloop, bloop, sort of, animation when you let your fingers up off the screen. And Scott Forstall hated it. He hated it. And his argument went like this. He said, “You know, that’s not fair to the designers of the apps, because they really didn’t design for what their apps would look like when you stretched them.”

      Frank: That’s super interesting. They didn’t have a say in what it’s gonna look like.

      Ken: That’s right.

      Frank: You’ve taken away their taste.

      Ken: And it’s an interesting aspect to what happens as you evolve a product. They would, then, for the subsequent version — but we would be shipping a version that added a new feature — multitasking gestures — and it would have to work with all the apps that were already in the world. Of course, there was a huge ecosystem by that point. So this was Scott’s argument — is that the designers — you’ve done something to the designers that they couldn’t really have accounted for in the design of their apps. Okay, so I got the chance to demo this to Steve, too. And I remember that Steve — what he did was, he had the iPad in his lap. So he was sitting like this, and doing the gestures, trying them side to side and whatever. And when he just discovered — by himself — this rubbery…

      Frank: End of list.

      Ken: …end of list animation, he did it, he did it again, and he didn’t look up. He said, “This is Apple.”

      Frank: Awesome.

      Ken: So it’s a pretty good moment for me.

      Frank: How did you stop yourself from like, doing the victory lap? Woo-hoo.

      Ken: He thought that it was — you know, sort of, tapping into the — excuse me, the little, sort of, whimsical aspect that went all the way back to, sort of, like the happy Mac on the original Macintosh, right? That it was this whimsical little animation that showed that the system has this playful character to it. And that was an aspect that he really loved. And so — and it also just goes to show that there could be disputes, even up at the highest level. Scott knew that I was very excited about this feature and wanted to show Steve, so he let me. And Steve was the one who had the final vote, and he sided with me in that instance.

      Frank: And do you feel like that slowed decision making down at all in the org, where basically, we’re just gonna wait for Steve to decide — so, like, why bother making a decision?

      Ken: But again, DRIs were responsible. You needed to bring him proposals, right? You know, you might think of that keyboard demo example — was, “Well, we were bringing him two keyboards and we wanted him to pick which one.” No, that wasn’t it. We were presenting him with a design we wanted to ship in the product. The design was going to have these two keyboards. He was the one who unpacked it, and to say we only wanted one of these. So no. And the point is, is that if you brought him shoddy work that was, like, you know, the equivalent of a shoulder shrug. “Steve, we’ve got five things, we don’t really know which one we think we like,” that was a way to…

      Frank: Never get invited back to a demo, right?

      Ken: It’s a way to not invited back to the demo. And that was the way that Scott Forstall, then, would have gotten blowback from Steve offline — to say, Scott, why aren’t you presenting me with solid designs? I’m not here wasting my time, I wanna see the full result of that bottom-up process. So that he could then give his top-down approval, disapproval — no, send this back for more work with specific feedback on what to change. That was the outcome of every demo with Steve. Approved, not approved, bring me something different next time — or not approved, give me these specific changes. It was one of those three things.

      Benefits of a blended background

      Frank: So Steve himself is, sort of, legendary for, sort of, fusing liberal arts and engineering thinking, right? And if you think about the classic Silicon Valley stereotype, companies are a lot more about, like, the pedigreed computer science engineer, right. Like, that’s the stereotype of, like, that’s what we’re looking for now. But your own background, and other people at Apple who’ve, sort of, had the valued liberal arts and engineering degree — talk about, like, what are the advantages of, sort of, melding the traditions? What’s an example of a decision that got made that was a better decision because you’re sort of…

      Ken: Well, I mean, it’s all the process of designing experiences for people that are useful and meaningful. Right? And I think that, how do we define what’s useful and meaningful? Well, we look to literature, right? We look to philosophy, right, we look to art, we look to the creative media, right? To decide what’s useful and meaningful. And so you know, I think — and you know, I didn’t know Steve well enough to know what he thought.

      But the culture that he helped to create, and that I found my place in that culture was — the part of the approach was that these devices are a part of people’s lives, righ? More and more now, to the extent that now, right, we think that there’s a problem with the amount of time that we’re spending looking at the screens, right, that we need to have apps and features on the phone to help us track, right?

      Frank: Too much screen time.

      Ken: Too much screen time, right? And so if we’re going to have this object, this device, these experiences that are so important to us, so deeply ingrained — well, then, it requires, I think, the care and attention and the thought about — it’s not just a technology artifact, it’s a social artifact, right? It’s a human artifact, right? And so that’s where liberal arts comes in. Yes, you do need to have the technological background to come up with the hardware, and the software, and the networking, and the services, to get everything packed together so that a product like this is possible. But if you — you know, you’re gonna ask well, what is it good for? You know, why do we do this feature rather than that feature? I think that yeah, that’s a liberal arts process.

      Frank: Tell the story, if you would, of how you guys arrived at the home screen app icon size, right? It was a fun liberal arts twist to this, right?

      Ken: Okay. So now, you know, going back to a phone that looks more like this, this is my original iPhone that I still have. So you know, this is the screen size that we were dealing with. Now, one of the — you know, again, now jumping back all the way to 2005, 18 months out from, you know, the product announcement. We were still in the early stages of trying to figure out, well, what is the home screen of apps gonna look like, and how is it going to work? And one of the fundamental questions that we had was, well, how big should the icons be?

      And again, I mentioned before, this apprehension of touching targets that were smaller than your finger. And we were still in the phase where we didn’t know how big on-screen objects should be. And we had some experiments, but this was still — we didn’t have a good handle on it. And so one of the engineers on the hallway had an idea. And his name was Scott Herz. He was doing work on SpringBoard, the icon launching program himself. And so, he had this idea, is that — I’m gonna make a game. It’s the first-ever iPhone game.

      Frank: iPhone game.

      Ken: Truly, because this is the point, we didn’t even have — all of our units still needed to be tethered to a Mac. We didn’t have standalone enclosures yet. So we were still at this phase where we had touchscreens that still needed to have a wire tethered to it. But still, we were trying to figure out what the ideal size is. And the game was the solution. And the game went like this. You would launch the game, and there was a minimal user interface. All it was a rectangle on the screen that was a random size and a random position. And the game was, tap the rectangle. And as soon as you did, it didn’t tell you if you succeeded or failed, because the idea was — just go tap the rectangle as quickly as possible. You tap the rectangle, the next one would show up at some other random size in some other random position on the screen. And the idea was to just go as quickly as possible, without, again, being, sort of, weighed down by the feedback of whether you were succeeding or failing. And you would get, then, 20 of them — and then it would give you your score, right? And so it was fun, right?

      Frank: Before “Angry Birds…”

      Ken: Before “Angry Birds,” we had the little…

      Frank: …was random rectangles.

      Ken: …rectangle going around. Now actually, what he was doing — he also wrote the software so that he was tracking, rectangle by rectangle, whether people were succeeding or failing. And also based on where the rectangle showed up on the screen. And within a couple — of course, the game was actually fun, right? I finally got 20 out of 20, right? We determined that if you made a rectangle that was 57 pixels square, that pretty much everybody could tap it 100% of the time, no matter where it was — again, since you were going quickly, you could tap it comfortably. And that number — he just, then, since he was working on SpringBoard, and it was his game, it was his app — he put that number into the app. He made the pixels 57 pixels square. And since that was a good number, we never changed it. And so that’s what wound up shipping on the iPhone.

      Frank: Yeah, I love that story, that it was, sort of, a game that led to it — as opposed to, “All right, we’re gonna do every possible pixel variation. We’re gonna bring people in to test it and we’ll see what works.”

      Ken: No, it was — again, he was the DRI for SpringBoard. It was his job to figure out how big the pixels should be. And he came up with a good solution, so we didn’t change it.

      Advice for getting into tech

      Frank: So, let’s switch gears a little bit and talk about, sort of, your advice for young people who are thinking about getting into the computer industry. Sort of, you know — liberal arts degree, computer science degree, what set of life experiences — like, what’s your general advice for people who want to join a tech company?

      Ken: I think it needs to be a mix. I think if you’re going to be a programmer — yeah, go write programs. I mean, the only way to get better at things is to do them. You know, and one of the wonderful things we mentioned, open source, you know, a bit earlier — the barriers now have never been lower to get involved. I knew that when I was a young person in college — I actually started in college in 1984 — I couldn’t afford a Mac, right? I wanted one.

      Frank: They were thousands of dollars…

      Ken: Thousands of dollars, there was no way…

      Frank: …back in 1984 dollars.

      Ken: There was no way that I could afford one. And so, now the barrier to entry is much lower. So if you’re interested in making projects — well, just go out and join a community and start making them. Or maybe you can even lurk in the community. You can download the software and try to make something of it yourself. So I think that — you know, again, if you want to do something, just start doing it. So that’s one piece of advice.

      And then the other piece of advice is, yeah, you do need to look at more than technology. Again, for the reason that I said a few minutes ago, which is — these technological artifacts that we’re making now have become so important to people, that if you don’t know anything about people, right? I don’t think that you’re going to be successful in the long term. And so, yeah, read books. Read books, study philosophy, go to art museums, learn about what’s beautiful and meaningful to you, answer those questions for yourself.

      You know, if you can answer those questions for yourself, it would be, then, hard as a product designer to then take on the responsibility of answering those questions for other people. Because that’s what you do when you’re a technologist in, say, a product company like Apple. You’re gonna be making decisions on products that are then gonna go out in the world and are gonna be affecting other people. Other people are gonna be putting those things and bringing them into their lives. And so how do you know what’s good?

      And so that’s a question that you should be prepared to answer for yourself. What do you like? And why? What are your goals? Why do you make a choice to make the product turn like this rather than that? And so it’s this combination of learning about the technology so that you can actually implement your ideas — but then you’ve gotta actually have good ideas. And again, it’s the liberal arts that provides the grounding for that.

      Frank: Super. And it’s counterintuitive in Silicon Valley, right? The suite of interview questions you typically encounter when you’re interviewing for jobs are about linked lists, and do you know TensorFlow, and can you program in Python or whatever — as opposed to what’s good, what’s beautiful?

      Ken: And really, you know, it’s unfortunate that there are so many questions like that. Well, obviously, linked lists — we’re still going to have need for those as we go into the future. But you know, the work that — much of the work that I did in my life, there was no way that I could have predicted, right? When I was handed, you know, a piece of hardware like this, and said, “Make a touchscreen operating system for a smartphone,” well, there were precious few examples that we could have looked at. And so how do you have experience in that thing? So again, I think getting flexibility and being able to answer the, sort of, more general questions about what you like, and what’s good, and what your higher-level goals are, because the technology is gonna change.

      Frank: And then, sort of, thinking about a company — like, how important do you think it is, if you’re thinking about joining a company, that there be a figure like a Steve Jobs, who has a trusted lieutenant like Scott Forstall? Like, is the absence of those ingredients — like, I’m not gonna join that company. Or, how universal is the Apple experience is another way of asking this question, versus how, sort of, specific to a set of characters and a time in history?

      Ken: Yeah, it’s a hard question. I mean, Steve was unique, right? And unfortunately, he’s not around anymore. And so I think it’s kind of a fool’s errand to go out and find who is the direct successor to Steve Jobs. You know, it’s just like, the questions are always changing. And so I think it’s a matter of finding a place where you feel comfortable, where you feel some sort of connection to what the organization is trying to accomplish — and that you like the people, and that you feel that you’re bringing something — you know, it’s this, kind of, this interesting contrast of both fitting in but then also I think providing more diversity.

      I mean, that’s an ongoing challenge for high-tech companies is that — again, as the products become more and more important for our culture, I think the people who are making the products need to be a better reflection of the world as it is, right? That it’s not just a bunch of computer geeks who went to, maybe, just a few high-powered schools that have good computer science departments.

      Apple’s culture of collaboration

      Frank: In your book, there’s, sort of, a couple key ingredients that you, sort of, distilled the Apple experience down to. Like, basically, in reflection, this is what made the iPhone team so productive. And you talk about things like collaboration, and taste, and decisiveness. So we’ll pick up, sort of, a few of these things as we, sort of, finish up the segment. So collaboration, right? Every company says “we have a collaborative culture.” What do you think made Apple’s unique?

      Ken: Well, it’s interesting that we were very, very good at combining complementary strengths, right? So, we had this human interface design team, and I worked very, very closely over time with a couple of the folks in there. Of course, there were only a few folks in there in total. And what we would do is — let’s say the example of me working with Bas Ording on the iPhone keyboard. And so, I was coming from the project primarily from an engineering direction, he was coming from the project primarily from a design direction. But Bas was pretty good at writing code, and I would fire up Photoshop and Illustrator. And so we would come up with these ideas, and we would complement each other.

      And you know, to the extent — again, you know, whatever you think of software patents, we got them for the work that we did in Apple. And one of the constraints that you have when you apply for patents is that you need to list the inventors. You actually need to be honest about who contributed to the specific invention. And so they would ask us, “Well, which one of you two came up with this specific idea so that we could write it into the claim language? And maybe if we’re gonna take that claim and move it to a separate patent, we have to know who to put as the inventor.” And Bas and I would look at each other and we would go “I don’t know, we both came up with it.”

      And so that’s the sign of collaboration — is that where the collaboration is so good that you don’t know where it begins and where it ends. You’re complementing each other so well that “we did it.” And there was no other way to describe it. And part of, you know, as a, sort of, concrete piece of advice — or maybe a way of describing that more at Apple is that — we didn’t have a lot of politics. You know, when Bas came up with an idea, or I came with an idea, it didn’t matter.

      Frank: It wasn’t a strong attribution culture, right? Oh, that’s his idea, and like, how dare you claim that that was your idea?

      Ken: And I can’t work on that. And now my manager is gonna get involved because now I’m not gonna get the credit for it and whatever. It just wasn’t like that.

      Frank: But you still had to have strong DRIs, right?

      Ken: Yeah. But that is also one of the ways that just made it clear about — you know, if I was collaborating with someone like Bas, or just some other engineer, you know, on the iOS engineering hallway. If I was the DRI for the keyboard, well, I was the one making the calls. You know, and as long as I kept making good calls, right? I mean, if somebody else had an idea that they really, really thought — they were gonna go to the mat, and they’re gonna say, no, I think Ken made, you know, the wrong call on this, yeah, they could buck that up the management hierarchy. But that was relatively unusual because, again, part of being a DRI is recognizing strong ideas that are coming from other people and including them in the work. And so that helps to describe some of the character of the collaboration that we had.

      Frank: Well, Ken, it’s been a fascinating conversation. Thanks so much for taking us inside the chocolate factory. Like, the chocolate factory did not have very many people. So I feel really blessed that, you know, one of those people made it out and is willing to lead the tour and talk to us. And maybe that’ll be the last question I ask you, which is — you know, the famously secretive Apple Corporation, right. Did you have to get their approval to actually write the book and tell the stories?

      Ken: Well, no, I didn’t. I don’t know if I was supposed to, but I didn’t. And I took a certain approach to it, which is that I think it’s a positive take on Apple. I loved my career at Apple. So I didn’t throw anybody under the bus, because there was nobody that I thought deserved it. And I limited myself to the Steve Jobs era, which is now, sadly or for good or for bad, passing into history. And again, I was one of the few people who had this perspective — this opportunity to be there during the time that some of these products were getting made.

      And so, you know, again, with my background being in history and being in the liberal arts, I thought that it would be good if I collected these recollections, while I still do remember them well, and tell the story. And so, I thought that it was really more of a personal story. And so, no, I didn’t. I was imagining that maybe I would ask for forgiveness if somehow they didn’t really approve. But I thought that I wouldn’t really run into trouble.

      Frank: Well, that’s great. Thank you for taking the time here, and for putting the stories down so they don’t fade into the mists of history. It’s been great having you, thank you so much.

      Ken: I’ve had a great time. Thank you.

      Frank: Great. So for those in the YouTube audience, if you liked what you saw, go ahead and subscribe. And then, in the comments thread on this video, let’s talk about things that you might wanna try in your own culture, now having listened to, sort of, Ken describe what it was Apple — what Apple did, sort of, what would work in your environment and what wouldn’t work in your environment? Would love to have a conversation about — how would you implement some of the ideas that we talked about in your own software development lifecycle. So see you next episode.

      • Ken Kocienda

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      Fintech for Startups and Incumbents

      Alex Rampell and Frank Chen

      In this episode of the a16z Podcast — which originally aired as a video on YouTube — general partner Alex Rampell (and former fintech entrepreneur as the CEO and co-founder of TrialPay) talks with operating partner Frank Chen about the quickly changing fintech landscape and, even more importantly, why the landscape is changing now.

      Should the incumbents be nervous? About what, exactly? And most importantly, what should big companies do about all of this change? But the conversation from both sides of the table begins from the perspective of the hungry and fast fintech startup sharing lessons learned, and then moves to more concrete advice for the execs in the hot seat at established companies.

      Show Notes

      • Discussion of how pure growth is not always desirable in insurance [2:17]
      • How some companies use complex data analysis to target the best borrowers [14:09]
      • Approaches that use social pressure to promote profitable behavior [25:26]
      • How incumbents might use multilayered branding to mirror the approaches of startups [33:30]
      • Turndown traffic and how incumbents can work with startups [39:04]
      • Advice for existing fintech companies regarding management and acquisitions [42:19]

      Transcript

      Frank: Hi, this is Frank Chen. Welcome to the “a16z Podcast.” Today’s episode is titled, “3 Ways Startups Are Coming for Established Fintech Companies — And What To Do About It.” It originated as a YouTube video. You can watch all of our videos at youtube.com/a16zvideos. Hope you enjoy. 

      Well, hi. Welcome to the “a16z” YouTube channel. I’m Frank Chen. And today, I am here with one of our general partners, Alex Rampell. I’m super excited that Alex is here. So, first fact — we both have sons named Cameron.

      Alex: We do.

      Frank: So, affinity there. And then two, one of the things that I really appreciate about Alex — and you can sort of see this from his young chess-playing days — is he understands fintech, and incentives, and pricing, backwards and forwards. And so, fintech has this hidden infrastructure on — how do credit card transactions work, how do bonds get sold, how are insurance policies priced? And there’s deep economic theory behind all of these, and Alex understands them all. So, you’re gonna have a fun time as Alex takes you through his encyclopedic knowledge of how these things are put together. And so, so excited to have you.

      Alex: Yeah. It’s great to be here.

      Frank: So, what I wanted to talk to you about is, I’m going to pretend to be in the seat of a — let’s call it an incumbent fintech company, right? So, I’m a product manager at Visa, or at GEICO. And I am looking in my rearview mirror, and there are startups in the rearview mirror. And I’m very nervous that the startup in the rearview mirror — exactly as the mirror says — objects in mirror may be closer than they appear. It’s, like — wow, they are catching up to me faster than I really want. 

      And so I want to understand, like, what are startups doing? How would they mount an attack on me, the incumbent? And we’re going to talk about, sort of, wedges they can use. And then that’s sort of the first half — like, how are they coming after me? And then the second half, let’s talk about, like — and what should I do about it? So, that’s sort of the premise for our — so why don’t we start with the attacks? Like, how would a startup come for me? And one way they come for me is they come after my best customers. So…

      Positive vs. negative selection

      Alex: Well, so this is the interesting thing about financial services, in general, because, you know, there’s a Sharp television hanging on the wall, and Sharp knows that they make more money every time they sell an incremental television. So, more customers equals more money — cause-effect. And the interesting thing is that for many kinds of financial services, that is not true, because what you’re really trying to do is assemble a risk pool. And the best example of this is insurance. 

      So, what is car insurance? Car insurance has good drivers, okay drivers, and bad drivers. And effectively, your good drivers and your okay drivers are paying you every month to subsidize the bad drivers. The same thing goes for health insurance. You have people that are always sick, you have people that are always healthy. And if you were an insurance company that only provided insurance for very, very sick people, or if you’re a car insurance company that only insures people that get into accidents every day, there’s no economic model to sustain that. You actually have to accumulate the good customers, and use them to pay for the bad customers. And the interesting thing about this is that from the perspective of the good customer, it’s not fair. And I’m not talking morally or philosophically, but just from a capitalist or economic viewpoint. 

      It’s like, okay, I want life insurance — and I eat five donuts a day. I just had a doughnut today. I don’t eat five a day, but I have one donut every Friday, as you can testify. And then I have a friend who goes to the gym five times a day, never eats a doughnut. That guy’s probably going to live longer than me. Hopefully not, but probabilistically, he’s probably going to have a better time than I am, in terms of life expectancy. So, why is it that we both pay the same rate? And that just seems unfair to him. Seems great to me, because he’s subsidizing me.

      Frank: Yeah, gym guy subsidizing doughnut guy.

      Alex: Exactly. Exactly. And that seems unfair. And then the startups can sometimes exploit that psychological unfairness, like, that feeling of unfairness. And it kind of does two things, because from the big company perspective, if you were to take away — think of it as a normal distribution. So, most people are in the middle, and they’re just going to live, whatever — to the average of 79.6 years, or whatever it is right now. Some people are going to live forever. They’re the ones that, you know, have the olive oil, go to the gym, and do whatever it is that they do that makes them live a long time, great genes. And then some people are going to die early. And from the perspective of the startup, if you can get all of the people that are going to live much, much longer, you’re going to be more profitable. 

      It’s the same thing for car insurance. If you can get all the people on the good end of that distribution curve, you’re going to make money. And then the nice thing is that if you’re starting a brand new company and saying, “Hey, I give you a loan, if you can’t get a loan.” Who’s going to sign up for that? People who might be bad. If I say, “I’m going to give you insurance, if you can’t get insurance,” who’s going to sign up for that? The people that are eating all the doughnuts. And that might not be very good. So, it actually has this nice, kind of, symbiosis between — if you do it correctly, you get positive selection bias, in that you establish a new criteria. Part of that new criteria is based on data, but part of it is based on psychology. 

      But the psychology is, “I’m treated unfairly, I want to be treated more fairly.” That yields a lower price for people, for [a] pretty demand-elastic product. So I say, “I can get life insurance at half the rate because I’m going to the gym? That sounds great. That sounds fair.” But to answer your question, what the incumbent might be left with, is not half of the number of customers. That could be the case — it could be half the number of customers — but it could be half the customers and all of them are entirely unprofitable.

      Frank: Right. They took over all the profits. They didn’t have to take all your customers, they just had to take the good ones.

      Alex: Right. So you’re actually — and if you just take — and the funny thing is that because it’s not — it’s not like I want to get, oh, “GEICO has X million customers, I want X plus 1 million customers.” You actually might want 1/10 as many customers as GEICO, because if you can just get the good ones — I mean, what if you give people a 50% discount, not a 15% discount, like GEICO advertises about — but a 50% discount on their car insurance. And these are the absolute best drivers in the country. How many claims do you have to pay out on the best drivers? You might have to pay out nothing, literally nothing. And if you have to pay out nothing — and there are these mandatory loss ratios for different insurance industries. So, I don’t want to get into that. 

      But imagine that, unregulated, you can pay out nothing — consumers feel like they’re treated very fairly, they’re rewarded for better behavior. This begets positive selection and not adverse selection — then you’re going to have the most profitable lending company or insurance company in the world, because it really is a unique industry where more customers is actually worse than less but more profitable customers, because each incremental customer is like a coin flip of profit or loss. Might generate profit, might generate loss. And that’s not true for the vast majority of industries. Like, Ford never sells a car saying, “Maybe we’ll lose money on this customer.”

      Frank: Right. Right. They just, like — I need everybody to buy a Ford F-150.

      Alex: They might…

      Frank: If you don’t buy an F-150, I need you to buy — this other thing, that’s the Expedition or whatever, yeah.

      Alex: They might lose money on the marginal customer until they hit their fixed costs, but they’re never going to have a coin flip of when they sell the car — “hmm, maybe we shouldn’t have sold that car.” But that’s what every insurance company has when they underwrite a policy. That’s what every bank has when they underwrite a loan, so…

      Frank: Yeah, auto insurance companies need to find people like me. I have this old Prius, right? First, it’s, you know — [a] hugely reliable car, and then I drive like a grandma because I’m optimizing for fuel efficiency. So, you know, I rarely go above 65. And so, like, rarely say — if I’ve never filed a claim. They need more customers like me, and that’s what drives the profits, right?

      Alex: Yes.

      Frank: Because there’s no payouts?

      Alex: Well, not only does it drive the profits, it actually subsidizes the losses — because there are a lot of people who are the inverse of you. And you’re paying for those people, and the transfer mechanism is through GEICO.

      Frank: Yeah. I saw an ad in my Facebook feed recently for Health IQ. And I think they’re doing something like this too, right? So, I think the proposition was, “Hey, can you run a mile in less than nine minutes? Can you bench press your own weight or something like that?’ There’s all these, like — ooh, healthy people. And is that the mechanism they’re exploiting?

      Alex: It’s exactly that. I would say the first company to probably do this on a widespread basis in fintech land was SoFi. And SoFi said, “Hey, you’re really smart.” They actually coined this term — they call it the HENRY — High Earning Not Rich Yet. Because if you looked at how student loans work, it’s like, everybody gets the same price on their student loan. It doesn’t matter what your major is, it doesn’t matter what your employment…

      Frank: Prospects.

      Alex: …thank you, what your employment prospects are. Everybody gets the same rate. You get this rate, you get this rate, you get this rate, because a lot of it is effectively underwritten by the U.S. government. And that’s not — so, think about it, again, from the twin pillars of psychology. Where, I mean, psychology of the borrower — like, how come I’m paying the same rate as that person who’s going to default? That’s just not fair. I’m never going to default. In fact, I’m going to pay back my student loans early. So, that helped.

      And then, again, positive selection versus adverse selection, because — and actually, refinance has this concept, in general. Because I would say, if you’re planning on declaring bankruptcy, or if you’re saying, “I’m going to join Occupy Wall Street and never pay back my loans, and I hate capitalism.” Why would you go refinance? It just doesn’t make sense, because you’re just going to default. So, if you raise your hand — and actually, it’s interesting, even on the other side, there are a lot of companies in what I would call the debt settlement space. And this is something that most people don’t know about. But if you listen to some interesting talk radio, you’ll hear all these ads for debt settlement. 

      And what is debt settlement? It’s saying, hey, do you have too much debt? If you call us, we will negotiate on your behalf and pay off your debts, and then you just owe us. And you, kind of, need this intermediary layer, because imagine that you owe $10,000 to Capital One, and you can’t pay it back. And you call Capital One, it says press 1 for your balance, press 2 to get a new card mailed to, press 3 if you don’t want to pay us the full amount and want to pay us less. Everybody is going to press 3, right?

      Frank: Everybody press — this is why they don’t offer that option. <laughter>

      Alex: They don’t offer that option, nor would they ever. However, on talk radio — and this is very big in the Midwest — like, you’ll hear, you know — Freedom Financial. They call Freedom Financial, and we will settle your debts for you. So, they call Capital One and say, “Look, Alex can’t pay you back. We’ll pay you $2,000 right now, and then you’re going to get rid of the loan.” Like, “Well, we’re not happy taking 20 cents on the dollar, but it’s better than 0 cents on the dollar. Fine. We’ll take it.” And then you owe Freedom Financial the 20 cents. But why do they feel comfortable underwriting that? Because you [raised] your hand, you know. You said, “I want to get out of debt.” And that’s positive selection bias right there. 

      Because people who are just deadbeats — because, you know, behind every credit score, if you think about how that works — it’s willingness and ability to repay. And the psychological trait of the willingness is, in many cases, as important as the financial constraint of the ability. So, if I owe $1 million to somebody, and I only make $100 a year — doesn’t matter how honest I am, I can never pay that back. It doesn’t matter how long — I mean, I could live 10,000 years and I guess I could pay it back. But otherwise, I can’t pay that back. But the willingness to repay is interesting. And that’s very important. 

      And that’s again, this kind of psychological trait, that’s captured in this idea of positive selection. So, what does SoFi do? They kind of, again, hit this twin pillar, which is — I want to only get the good customers. I’m going to reprice them and steal them from the giant pool, that, again, normal distribution. These are the losers, these are the whatevers, and these are the people that you have no risk on whatsoever. Let’s steal all of these people over here. And it makes them feel good. It’s a better marketing message, because it’s differentiated. Like, how do you compete with everybody? It’s like, “Hey, we’re just like Chase but smaller, and a startup, and not profitable. And you probably shouldn’t trust us.” Bad marketing message. Good marketing message is, “You’re getting ripped off. We’re going to price you fairly. Come to us.” So, SoFi did this for lending, and then…

      Frank: And what did Health IQ do?

      Alex: So, Health IQ did this for health, really for life insurance. So, they started off with a health quiz. Because, I mean, it seems almost self-evident that healthy people are healthy — I mean, it’s a tautology. Like, healthy people are healthier than not healthy people. But can you actually prove this from a life expectancy perspective? So, they started off with just recording data, and then building a mortality table. And it turned out that, you know, what I would assume is a prima facie case, turned out to actually be correct — which is, these healthier people do live longer than not healthy people. And then they turn that into both a positive selection advertising campaign, which differentiated them from a brand perspective — but also left them more profitable. 

      So, what they do is they say, “Yeah, can you run a nine or an eight-minute mile? Can you do these things to prove that you’re better than everybody else?” And why is that important? Well, from their own balance sheet or profitability perspective, they want to get these good customers. Versus, you know, a brand new life insurance company that said, “Hey, life insurance takes too long to get, it’s a big pain, and it’s expensive. We’ll underwrite you on the spot in one minute, no blood test.” That’s gonna be adverse selection. That’s like, ooh, I think I’m gonna die soon. <Right.> I want to get — and everybody rejected me for life insurance. I’m going to that company, as opposed to here, they’re only getting the customers that kind of hit — they think they’re going to hit the underwriting standard, which is great. They think it’s fair. So, it’s a differentiator from a brand perspective, and then it turns out that again, each marginal customer in insurance is kind of a coin flip. They’re getting a weighted coin, because they’re only getting people on the far right side of this normal distribution.

      Using data to find the best customers

      Frank: So, wedge number one is exploit psychology, right? Positive selection, rather than negative selection. And what you’ll end up with, because of this sort of unique dynamic of the fintech industry, is you’ll end up with the most profitable customers. What’s wedge number two? We’re going to talk about, sort of, new data sources, and what startups can do to sort of price their products smarter than incumbents.

      Alex: Right. So, imagine that you have a group of 100 people, and of the 100 people, half of them are not going to pay you back. So, think of this as the old combinatorics problem of, you know, bins and balls. You’ve got this giant ball pit, you scoop up 100 balls in your bin, and half of them are going to be bad, half of them are going to be good. So, what’s a fair rate of interest, if you’re a lender, that you have to charge this whole bin, if half of them are going to default, and you assume that you can’t lose money? The answer is going to be 100%.

      Frank: Oh right. Because half of them, you have to make up for all the deadbeats.

      Alex: Right. So, half of them — you know, you lose all of your money, half of them, you double your money. So you’re back to square one.

      Frank: Now you’re even.

      Alex: Now you’re even. So, the problem is that that’s not good. Because well, in the United States, you can’t charge 100% interest. It’s regulated.

      Frank: Right, right, illegal, step one.

      Alex: It’s called usury. There are other parts of the world, again, illegal. Step one. Europe — so, that’s a problem. But, what if you can use different data sources to — again, it’s not positive versus adverse selection, as in for some of the insurance companies, but it’s saying, can I collect more forms of data? So that instead of saying the only way that I can make my operation work is to charge an interest rate which actually turns out to be illegal — can I come up with more data sources that effectively — even though discrimination sounds like a terrible word, and it’s certainly used in that construct. If you discriminate against criminals, that’s fine. I mean, some of the people that try to take advantage of lenders are actual, like organized crime. You don’t want them in your bin, you want to throw them out. 

      How do you take more data sources and actually start measuring this? And the interesting thing here, and it’s somewhat unfortunate — but you have a giant market failure happening in many different regions of the world. Because in the United States, like, the top interest rate that you can charge — it’s regulated on a state by state basis — but Utah has a 36% usury cap. So, a lot of people export that cap. That’s a lot less than the 100% that I was mentioning. And there are lots of ways of, kind of, gaming that system. You charge late fees, and you charge this fee, so it actually might end up looking more like 100% or 200%. So, you can’t charge more than 36%. 

      And then you actually can’t use certain types of data, if they are prone to having an adverse impact. So, if you think about how machine learning works — I always kind of describe it somewhat over simplistically as linear algebra, where I have — here’s every user that I’ve ever seen, here’s every attribute that I’ve ever measured. And what I’m looking for is strange correlations that I can’t even explain. So, I’m going to ask you — I’m not even asking you a lot of these things. It’s like, how long did you fill out this field for on my loan application? Did you enter all caps or not all caps? Like, just all of these different things.

      Frank: Did you take the slider on “how much do you want?” and jam it all the way to the right. All of these things.

      Alex: Right. I can ask you, do you have a pet or not? That might be interesting. I don’t know if that’s a leading indicator of defaults or not, but I want to collect all these different variables. And then at the end of the day, I’m going to see “default” or “not default.” That’s the output. And then I’m going to see what’s correlated with that. And it’s a little bit of this, it’s a little bit of that — I can’t explain it, but the computer can. Now, the problem is that in the United States you actually can’t do this, because it might have an adverse impact. And what does an adverse impact mean? There actually was outright and terrible discrimination in lending in the United States. Well, there’s, unfortunately, terrible discrimination in many things in the United States — but lending was one of several, or one of many. 

      So, imagine that I said, “Are you married or not? Oh, you’re not married? I’m not going to make you a loan.” Well, that’s illegal now. “Are you this race? Oh, I’m not going to make you a loan.” Well, that’s illegal now. So, what did people do to get around — the people that were actual racists? Or actual, like — maybe they weren’t racist or discriminatory at heart, but they were picking up on cues. They’d say, “Oh, what part of town do you live in? Oh, you live [in] that part of town. Well, that’s like 100% correlated with this race, or this gender, this, that. I’m not going to make you the loan.” So, the law was strengthened. So, there’s a law called Fair Lending in the United States. And then one of the components of it, is this idea called adverse impact. It’s different than adverse selection. It’s saying, I don’t care what you said you did for why you rejected Frank for a loan. If it turns out that everybody in your reject pile has a disproportionate, you know, gender ratio, race ratio — something like that — I’m going to assume that your underwriting standards are having an adverse impact.

      Frank: So, you as a bank, couldn’t say, “Hey, look, I asked him if he had cats, and I’m using that to make the loan decision.” If it turned out that having cats was correlated with being a particular race, they couldn’t use the “cats” answer to deny you a loan.

      Alex: Correct. Because that was — and in all fairness to the law, this is what people use with your geography. “What zipcode do you live in? Oh, you live in that zip code?” One hundred percent you are a member of this particular race, and the intent all along was to discriminate against people of that particular race. But now, instead of using loan officers that use — you know, God knows what to decide — do I want to make you the loan or not? You’re using a computer, you can look at the code. 

      So, I think there is a lot of — there are some anachronistic laws that have to catch up here. But let’s take an area outside of the U.S. to answer your question, where perhaps you don’t have interest rate caps. Because, you know, the thing that a lot of people say, “Oh, you know, 200% interest is terrible. Five hundred percent, that sounds awful, you should go to jail for that.” But what does APR mean? APR stands for annual percentage rate. And what if I’m giving you a four-day loan? So, I say, okay, I’m gonna loan you nine dollars right now, you don’t look very trustworthy. I want you to pay me back $10 on Monday.

      Frank: Yeah, that doesn’t sound so bad. It’s a buck, right? Yeah.

      Alex: Yeah, it’s like, you’re gonna pay me a dollar. But what is that on an APR basis? That’s like 9,000%. I made that up. But it’s probably about that, right? Because it’s 10% every four days,or every three days, 10% every three days — and that accumulates. Like, that’s a lot of money or a lot of interest on an APR basis. But it’s the wrong metric because, effectively, it’s like trying to figure out what your marathon time is based on your 100-meter dash. Like, the winning marathon time would be an hour, and that’s not true. We know that nobody…

      Frank: Nobody can do that.

      Alex: …can do run a two-hour marathon, right now. Yeah. So, maybe Angela can.

      Frank: Maybe Angela.

      Alex: So, there’s a company that we invested in called Branch. And what they’re doing is, they just collect every form of data possible, and they look for these strange correlations. And the interest rates on an APR basis might be high, but they’re really charging, like, a dollar to the lenders.

      Frank: And these are small loans, right?

      Alex: They are very, very small loans. So, I loan you — and actually the other interesting — like, one of the nice data points that they’re accumulating over time that is a really interesting idea, I think — it’s not new. In fact, it’s almost “Back to The Future” old, where they loan you a dollar, if you pay it back, they loan you two dollars. If you pay it back, they loan you four dollars. If you pay it back, they loan you $10. And they ladder up your credit, and they keep that information proprietary to them. Because induction turns out to be a pretty good formula for figuring out not so much the ability to repay, but the willingness to repay. You’ve established a pattern of willingness to repay. But they also look at “where were you today?” And again, you provide all of this information in order for them to crunch this — in order for them to give you a loan at, ideally, a lower rate. Because the more information — because it’s kind of twin pillars, right? The less information we have, the higher the rate that we have to charge. Not because we’re evil, but because otherwise, you’re going to have a market failure, like you have in lots of the…

      Frank: Yeah, the bin and ball problem, right?

      Alex: Exactly.

      Frank: Because you have no idea how many deadbeats.

      Alex: Exactly. And if I don’t have any idea, I either have to charge a high rate or not charge anything at all. And “not charge anything at all” doesn’t mean, like, everybody gets a 0% loan. It means I don’t make any loans. And like both of those are bad outcomes. The better outcome is, you accumulate more data, and you figure out “here are the good people — let me not accept the bad people.” Because again, the way that the good people end up paying more money is if the company starts accepting more bad people, because it goes back to what I said at the beginning — which is, more customers, in this unique industry, often is bad if you don’t understand how to select them correctly. And for many of these new-fangled lending and insurance companies, the default customer is going to be adversely selected. Because if you’re a new lender, and you have no underwriting standards, basically, you’re advertising free money — never pay us back. And those are the people that will be attracted to you, both the criminals and the non-criminals in droves.

      Frank: Yeah. So, this is, sort of, startup attack wedge number two, which is — I’m going to generate a new data source that allows me to price my product in a way, or reach a customer that a traditional company would never even try, or they don’t have the data source, so they have the bin and ball problem. So, what are the types of data that Branch went to go get to try to figure out — should I give you a loan of a dollar or two?

      Alex: Well, the other type of data — so, Branch was somewhat unique, in that they said, “We’re going to get data from your phone.” And it seems odd — it’s like, most lenders in the developed world — or not developed versus undeveloped. It’s really, like — with developed credit infrastructure. If they look up…

      Frank: If there’s a credit bureau.

      Alex: They look up your credit report. If it’s good, they make you a loan. If it’s bad, they don’t make you a loan. It’s actually not that hard. And there are all sorts of nuances that you can layer on top, but this is how it’s been working for a long time in the United States as an example. Whereas there, it was like, “Okay, where did you work today? Did it look like you worked today?” So, it was stuff like that. And even like, how many apps do you have on your phone? Like, weird stuff that you would never assume actually has any kind of indication of willingness or ability to repay, but in many cases, it does. Like, are you gambling? Well, if you have a gambling app on your phone, you’re probably gambling. Maybe that’s good. Maybe it’s bad. It’s actually not making human judgments — and it’s also not looking at any one of these unique variables as a unique variable. It’s looking at them in concert, and then correlating them with these outcomes, so really observing the outcomes and then linking them back to all of these different inputs.

      Frank: Yeah. I remember talking to the team when I was researching my last machine learning presentation, and the fascinating things that I found were — if you’ve got more texts than you sent, you were more creditworthy. If you had the gambling app, you were more creditworthy, rather than less — which is not kind of what you would expect. If you burn through your battery, you were more likely to default, right? So, like, all of these things where a human, or loan officers, would never really guess, right? And they probably would guess the wrong way because they wouldn’t guess…

      Alex: Because many of them are counterintuitive. And then many of them, they’re not unilateral. Like, so it’s not just — I mean, I don’t know. But it’s not just the battery thing. It’s the battery thing with this, with that, with that. If you think…

      Frank: Right, different combinations, right?

      Alex: It’s like, you know, humans can only really observe three dimensions plus time — so I guess four — and these are, you know, 9,000-dimensional problems. So, it’s much, much more challenging for humans to really grok.

      Influence of social pressure

      Frank: Yeah. Got it. So, that’s the — sort of the second category of attack. Which is, you generate a new data source, and then that allows you to price or find customers in sort of a more cost-effective way. Let’s talk about the third, which is around, sort of, fundamentally changing behavior. So, why don’t you talk about — maybe Earnin is a good example of this?

      Alex: Yeah. So, if you assume that humans are static — so they’re born — both of our Camerons were born, and their DNA is set upon birth. Maybe it changes a little bit with some mutations from some gamma rays here and there. But it’s set upon birth, and then human behavior never changes. And that’s one way of looking at things. Then you think about adverse selection versus positive selection. Good drivers are always good drivers. Bad drivers are always bad drivers. Let’s just get the good drivers. 

      So, the other category — and it’s not just that these other two groups don’t do this. But if I look at a company like Earnin, most payday lenders are reviled, because they charge high fees, they don’t educate their borrowers very well. Now, it actually provides a valuable service, because if I’m getting paid next Friday, but my rent is due today, and I don’t have money, do I want to get evicted? No. I want to get paid right now, and the only person that does this is the payday lender. But the payday lender is competing with other payday lenders for advertising in the local newspaper, or something. And if they’re able to rip me off more, not because they’re evil but because they have to afford the advertising spot — they’re now incented to do so. So, it’s a vicious cycle. 

      So, let’s talk about Earnin. So, what Earnin does, is they say, okay — we know that you’ve worked this long. So, again, new data source — because the phone’s in your pocket, and you work at Starbucks, and you’re getting paid hourly, and we’ve seen the phone in your pocket, or in your locker in the Starbucks office, you know — by the barista counter, for eight hours. So, you worked, we saw your last paycheck hitting your bank account, we know that that’s where you work. We’re not taking your word for it. We have real-time streaming information about this. And now we will give you your money whenever you want. Not money that you haven’t earned yet, but money that you have earned, but you actually haven’t gotten paid for yet. And then you can tip us. There’s no cost. If you want, you can tip us.

      Frank: No interest, no fee, no — huh.

      Alex: Nothing. If you want to pay us nothing, that’s fine. I mean, we would appreciate it if you pay us something, because obviously, we’re providing a valuable service for you. And then you can even give tips for your friends. There’s this community that’s really emerged of people on Earnin. And actually, if you look back at different business models — but this idea of microfinance, in general. So, if you think about Muhammad Yunus and what he did — this idea of, can you encourage people to pay back loans using social pressure? So, again, not adverse selection versus positive selection, but actually trying to force everybody down positive behavior.

      Frank: Yeah. Let’s get the community to encourage repayment.

      Alex: Right. Because then, saying — or, like, let’s get the community to encourage people actually driving safely, because there’s underwriting at the time of admission. There’s underwriting based on ongoing behaviors. So, like, many of the car insurance companies that are brand new are saying, “We will re-underwrite you, like — yeah, if you drive like Frank when you signed up, great. But now you switched into, like, race car driver mode, and you were trying to hack us, but we’re actually monitoring your speedometer at all times. So, guess what? You got a higher rate now. So, that might encourage you to drive safely.” 

      If I’m Frank, and I drive safely in my Prius, but then I decide — and then I got a really good rate on my car insurance as a result. And now I’m like “Aha, I gamed the system, now I’m going to drive like a maniac.” Well, the nice thing is that you can make underwriting dynamic, and you can say, “All right, we’re actually going to re-underwrite you every day.” So, we have the positive selection to try to attract the Franks. We have the continuous evaluation to try to encourage the right behavior, post-Frank signup — and also to stop the gamification of — it’s like, I’m going to pretend to be safe and then be like a maniac. But then how do you actually get — what if Frank was a bad driver initially? Doesn’t fall into my positive selection loop, but I still want to try to make Frank a better driver.

      Frank: Yeah, if I can turn him into a good driver, he’d be profitable. So, what can I do?

      Alex: Right. Because that’s the flaw with, kind of, wedge one and wedge two, of like, creaming the crop. Really wedge one, which is we’re going to cream the crop. We’re going to do what SoFi did, we’re going to do what Health IQ did. I mean, it’s a great strategy, but the rest — again, if you assume that it’s all nature and there’s no nurture, then perhaps there’s nothing you can do. But if you can actually try to nurture better behavior, you actually see better — you do see better behavior, and then the profitability goes up. And the interesting thing there is that you’re still finding mispriced customers, but you’re actually helping turn them into correctly priced customers. 

      So, you know, somebody, like a bank would turn away that customer, and say, “We don’t want them because they have a 500 FICO,” which is really bad. And then you have to figure out — and as with all of the new startups that are saying, “We only want the best customers — we want to leave the banks with the bad customers.” But it’s kind of the twin pillars of — can you identify something that’s below that credit score, or below that driving score, or something? And then can you encourage positive change? And if you can, then you can start actually creaming the crop of the bottom half of the customers. Not even the bottom half, it’s the customers that are just neglected, because nobody wants to underwrite them. And then you do that, you take them on, because you have a secret to change their behavior.

      Frank: Right. You’re seeing a lot of companies that, sort of, are using behavioral economics research to figure out, “How do I nudge people into better behavior?” And so, this would be an example of how you’re trying to change behavior to get the profitable customer.

      Alex: Right. So, you know, there is one company in the lending space a while ago called Vouch — I think ultimately, it didn’t work. But when you apply for a loan, it actually, kind of, taps your social network, and it requires that they do a reference for you. Either a reference, in terms of like, yes — Frank is a good customer, you can trust him. And even kind of a co-commit. So, I’m getting a loan for $1,000, and you say, “Yeah, Alex is okay.” Or I’m saying, “Frank is okay. And if he doesn’t pay you back, I will put $100 in, because that’s how confident I am.” And it’s not all $1,000, but it’s $100. 

      And then you’re my friend — I go bowling with you. We go take our Camerons out together. And if you don’t pay back this $1,000 to this, kind of, faceless, large, evil corporate entity — not really — but if you don’t pay that back, I’m on the hook for 100 bucks. I’m not going bowling with you anymore. So, there are other things that are really interesting to try to encourage the correct form of behavior, when — and, actually, part of it is just making it personal. Like, this was the whole Yunus theory. Which is, if you are, kind of, held accountable by your peers, that is so much more powerful than getting a collections call from Citibank. Like, you’re like, “Ooh, that’s the collections number?” iPhone block. Done. But how am I going to block my friends out?

      Frank: Right. If Alex calls me and said, “You really got to pay that loan back, otherwise, I’m out 100 bucks,” right? That’s much more powerful. I mean, this has worked great for Omada Health in a different domain, right? Which is, if you are trying to get a pre-diabetic patient not to get diabetes, the most effective thing to do is lose something like 6% or 7% of your body mass. And the way they do it is they get you into a group. They mail everybody a scale. Everybody sees your weight in the morning, right? Like, that’s a powerful motivator.

      Alex: Yeah, I mean, this stuff — psychology is very powerful. So, there are a lot of tricks that you can use here. And if you understand the impact of them, you actually have to reassess your entire branding and customer acquisition strategy.

      Frank: Right. Right. All right. So, remember, I opened up pretending to be the product manager at Visa. And now we’ve gone through all of these three categories of how the startups are coming for me — and like, I’m starting to sweat here, right? They can come and get my best customers, they can generate new data sources that I would have a hard time doing. They can actually even go after sort of worst customers, change their behavior, turn them into profitable customers. I’m scared now. Like, what in the world should I do? Like, you’re in my seat — you’re the head of innovation, or head of strategy, or head of digital at one of these big fintech companies — what should I do with respect to startups?

      Alex: Well, I think it’s actually very hard for a company that’s trying to be all things to all customers. Because, if you look at what SoFi is — look at SoFi’s brand. Brand is, you know, we are the high — like, if you’re great, you’re good enough for us.

      Frank: If you’re HENRY, right?

      Alex: If you’re a HENRY, you’re good enough for us. Health IQ. If you’re healthy, you’re good enough for us. So, on that sector of the curve, you know — how does GEICO say, “Hey, if you’re a good driver, go to this special part of GEICO. If you’re a regular driver, you still save 15%. If you’re a bad driver, and you had a DUI, well, we can cover you over here.” It’s lost in this, kind of, giant GEICO gecko marketing message. So, in many cases, it actually helps to have sub-brands and divide this up, which is somewhat anathema to a lot of companies that want to say, “How do we get as much efficiency and synergy as possible? We’re going to have one overarching brand.” And you know, one of my favorite examples of this — kind of, different industry — but the highest end of the highest end of jewelry is Tiffany & Co. Or, one of the highest and the highest end.

      Frank: Beautiful, beautiful rocks.

      Alex: And for a long time, it was owned by Avon.

      Frank: No, really?

      Alex: You know, the Avon lady, Avon. And if Avon bought Tiffany, which they did, and they said, “Okay, we’re gonna rebrand Tiffany & Co. as Avon,” like, that doesn’t work. Like, you’re not going to get 80% gross margins on whatever they sell at Tiffany & Co for…

      Frank: Breakfast at Avon’s just doesn’t have quite the right ring.

      Alex: It doesn’t work. And then for Avon to say, “Okay, you know, the door-to-door salesperson or sales lady with the pink Cadillac that’s going around, like — we’re now going to have her push, you know, $2,000 bracelets, as opposed to the normal $10 fare.” Like, that’s not going to work either. But it actually can make sense, if you want to just appeal to more customers, you have different brands, and you don’t want to all suck them together. So, you can imagine instead of having, you know — GEICO could be your generic brand, but then you could have — I think I mentioned this to you once before, a friend of mine is Mormon. Doesn’t drink alcohol, and says we should have Mormon Insurance for cars, because it’s just totally unfair. Again, going back to the psychology point, like — why is it that I’m paying for the drunk idiot that goes through the stop sign? I don’t drink, I can prove that. I will never drink, I have a million friends just like me that will never drink. We should all get car insurance — we should all get a 40% lower rate. 

      Do they think of GEICO when they go there? Maybe they could. But it could be like, Mormon Car Insure- — sorry, I’m not good at branding. But you could have a separate brand for all of these separate subgroups, and have the same underlying infrastructure behind all of them. But, again, part of this is just how do you brand and how do you market effectively? Because if you look at the efficacy of Health IQ ads, or the efficacy of SoFi ads, there are so much higher — because again, you have this large group of people — or in many cases, small but valuable groups of people — that feel like they’re being treated unfairly. So, yeah, GEICO is save 15% on auto insurance, click here. Mormon Car Insurance, advertises to LDS members in Utah, shooting fish in a barrel — that’s going to have a dramatically higher click rate. And then many of these products are also very demand-elastic. So, I’m not saying save 15% on car insurance, I’m saying save 80% on car insurance. It’s very easy to do. Click here, positive selection bias. That’s going to work better than GEICO, but we also have something for Mormons, too.

      Frank: Right. Yeah, the goal is to find the LDS’ers and the hypermilers who are really safe, etc., etc., right? And so it’s very counterintuitive, because if you’re at a big company, you’re thinking scale — how do I get the next increment of revenue, growth, or profit? And you’re saying, actually go the other way. Don’t try to make your single brand bigger. Try to think about a dozen sub-brands, each going after sort of the perfect market for them. How do you positively select into a sub-market?

      Alex: Well, the other side effect of this is that, you know — part of the asymmetric warfare that some of the startups have is that, if you wanted to kill GEICO, you wouldn’t steal 100% of their customers. Because if you did that, that would almost be too obvious. You’d steal 20% of their customers, but only the good ones. So, imagine that GEICO could actually devolve, or evolve — depending on your point of view — into 10 sub-brands. There’s no more GEICO. But it’s just, like, the 10 sub-brands basically select for the right types of customers, or even help judge and improve behavior from other subsets of customers. And then expel the 30% that are just bad news. And if you can expel the 30% that are bad news, you might say, “Okay, well, all of this de-synergy of going from 1 brand into 10 sub-brands — well, that was idiotic, because now I have fewer customers.” But actually, no, it isn’t. Because you might have fewer customers, but it’s not like selling widgets, you’re selling probabilistic widgets — where, in many cases, you have negative gross margin when you sell a widget. So, it’s important to figure out how do I get the good ones, keep the good ones, and then get rid of the bad ones?

      Branding for incumbents

      Frank: Yeah. So, that’s one strategy, which is, sort of, sub-brands — and, sort of, customer segmentation. What if I’ve been told by my management team, “Go find a bunch of startups to work with,” right? Sort of, somehow figure out a marketing or co-selling relationship so that we can start experimenting with some of these new models, and we can keep an eye on the startup community. So that maybe, you know, we can put ourselves in the best place to buy them if it turns out working? Is there a way to do that?

      Alex: Well, there are many ways to do that. Probably the easiest way that is often counterintuitive for a lot of big companies — is I call this the turndown traffic strategy. So, Chase turns down a lot of people for loans, either because —again, it’s the bin and ball problem — where it’s like, well, you might be good, you might be bad. Sometimes it’s not even that. It’s like, we think you’re good, but we just can’t profitably underwrite a $400 loan. But Chase has all the traffic. So, what is turned down traffic? It’s saying, “Okay, we rejected you. Hey, here’s a friend that you might like.” So, this is not cream of the crop — this is the bottom tier on the ingestion point for a big financial institution saying, we don’t want you — which is kind of a mean thing to say. A way to ameliorate that potentially is saying, “We don’t want you because we’re not smart enough to — hey, sorry, we’re working on it. All our systems are down. But here’s a great startup that does.” Now, why would you send customers to a startup? Well, the number one thing — GEICO spends $1.2 billion a year on advertising. It’s really hard to compete with that…

      Frank: A lot of spend.

      Alex: …from a — so, if I could not spend a dollar of advertising, but give 90% of my net income to GEICO as a startup, I still might make that trade. I mean, we don’t always like this, because we want to see — do you have your own acquisition strategies, your own acquisition channels — you’re not dependent on the big company. But from the big company’s perspective, turndown traffic is often brilliant. Because it’s saying, “Here’s somebody that knows how to underwrite better than we do, or more profitably than we do. We’re going to send our customers” — we said, you know, otherwise, what happens? 

      And this is what I think Amazon got right in an area where everybody else got this wrong. Amazon said — okay, you’re on Amazon’s website, and you’re looking at the “Harry Potter” book. And then right next to our “Harry Potter” book is an ad for Barnes & Noble for the “Harry Potter” book. Barnes & Noble is like, “This is amazing! We can buy ads on Amazon’s website! They’re so stupid. We’re buying ads, it’s stealing their customers.” But every time you click on that Barnes & Noble ad, Amazon made a dollar. It’s 100% gross margin — they share that with nobody. There’s no COGS on that. 

      And then they can use that dollar of pure profit to lower the cost of their “Harry Potter” book, which actually made more people want to go to Amazon to look for “Harry Potter” than go to Barnes & Noble — that said, we’re locking [you] within our walls. It’s like a casino with no clocks. And we’re gonna pump oxygen in. Because what a lot of big companies don’t get is that Google is just one click away. Like, why give all the excess profits to Google, when I go to Chase, I get turned down for a loan. And then I go back to Google, and I say, “Where else can I get a loan?” Well, Chase should be sending you there. And actually, they’re starting to do this. So, that’s one strategy that I think has a lot of legs.

      Frank: Yeah, so turndown traffic. That’s super interesting. Look, you spent all the money to bring them to your site, and otherwise, you would have just lost them, right? That sort of sunk cost.

      Alex: Exactly.

      Acquisitions and team-building

      Frank: So, you get something out of it. That’s fantastic. Well, why don’t we finish this segment out? I want to do a lightning round with you, which is — I want sort of, you know, instant advice for somebody in this seat. I’m an exec at Visa or GEICO. And so I’m going to name a category and you sort of just — of how to deal with startups, and you can react to it. All right. So, category one is, you should always invest super early — as early as you can into a startup.

      Alex: So, again, remember adverse selection versus positive selection. So, I would say, the companies — so, this is what you have to get right. Which is, if you take nine weeks to make a decision, and like, you know, we’ll decide within a day — or if Sequoia or Benchmark or some other great venture capital firm will decide within a day — like, you’re not going to get good deals if you take nine weeks. So, it can be very, very important to invest early — but, like, the best things always seem overpriced. Like, this is something that we’ve learned, and it’s the same thing with underwriting your own customers. Which is, like, if something is too good to be true, it probably is. So, some of the best things are actually very expensive.

      Frank: Yeah. All right. Just given those dynamics, just wait for the later rounds. Let all the venture guys take all the risk, and then, like, you plow in late, that should be a nice strategy.

      Alex: I think, in general, that’s probably a better strategy. But again, saying, like, “Ooh, we’re getting a great deal on this one.” That’s probably — then, you know that you’re the adverse selection source of capital, as opposed to — okay, here’s something I can’t believe we’re paying this much money for it. We have to fight our way in. There are 10 other people that want it. You probably know you’re onto a good customer, if you will, or a good investment.

      Frank: All right. Partner with as many possible startups as you can, because you don’t know who’s going to win, so let’s open up a marketplace. A hundred startups that I have — either turned on traffic relationships or something.

      Alex: I think actually that does make sense. I mean, there should be some kind of gating item to make sure, like — maybe not 100, but how do we stay close to different models that are working well? Because the main advantage that the incumbents have — again, it depends on lending or insurance — but it’s typically something around cost of capital and something around distribution. So, if you have both of those, and you’re not using it to the fullest extent — like, you turn down a lot of customers — you should try to find an intelligent way of using this and using — that’s your unique thing. Like, venture capital firms don’t have that. I can’t fund somebody and send them a million customers tomorrow, but GEICO could. But you can’t do that 100 times, you can probably do that some sub-segment of times, according to how much, you know, additional traffic — or whatever it is that the unique advantage that you want to bring to bear.

      Frank: All right. Now, on M&A strategy. M&A strategy one — buy super early before it’s proven to work — because, presumably, the prices are lower. So, M&A strategy early — focus on early-stage companies.

      Alex: I’m a big fan of what Facebook’s done with M&A, and I encourage everybody in pretty much every other industry to do this. So, Facebook has two formats for M&A. One is, we buy the existential threat that could kill us, and we price it probabilistically. So, surrender 1% of our market cap to buy Instagram. That was way overpriced.

      Frank: Instagram, that’s a good example. Everybody said that, they said, “Why are you…”

      Alex: But one — like, there’s a 1 in 100 chance that this is going to be bigger than Facebook. We should probably surrender 1% of our market cap. WhatsApp, 7% chance, or whatever it was. I think it was 7% of Facebook’s fully diluted market cap — was spent on WhatsApp. These were brilliant acquisitions. Oculus. I mean, Oculus hasn’t turned out the same way that WhatsApp has, perhaps — but, like, same idea. It’s like, this could be the new platform. If we don’t buy this, and Apple does, we are subject to their random whims and fancies. So, that’s category one. Category two — and this is super counterintuitive for a lot of companies — buy the guys that failed trying. Because they had the courage and the tenacity to try to go and build something new. And that’s what you want in your company as well. And then — this is the most counterintuitive part is — like, take the person that failed and put them in charge of the person that was successful. And that’s breaking glass.

      Frank: For a big company, that’s so hard. You reward your execs on success, not on failure.

      Alex: Right. But in many cases, it’s like, you have a big company that’s been trying to build this thing for 10 years. And if they build it, they will get 1 billion customers, because they — I’m making that up. They have the distribution. Then you have the startup that actually built the thing in, like, a week — and they built it for $1 million. And that would take the big company, like, $1 billion dollars and 10 years to do. But like, “Oh, the company failed. Oh, that’s a bad company. These are bad managers.” But actually, you want to take them and put them in charge.

      And the joke that I always make is, like — if Amtrak buys Tesla, the worst thing that Amtrak could do — because Amtrak is probably more profitable than Tesla, at this point. But if Amtrak were to buy Tesla, the worst thing they could do is say, “Okay, all of you Tesla bozos, you work for us.” But the whole point of a lot of this other form of M&A is — you’re really trying to buy products that you can push into your distribution. And you’re trying to buy talent that wrote the products, that built the products, that understand that. And the only thing that they needed, the only gap between them and actual huge success is distribution, which these big companies have in droves.

      Frank: Yeah. So, that makes perfect sense. Maybe just a piece of advice on how to actually make that happen. Because you have this dynamic, where you’re a big company, you just bought a failing startup, right? You have all of the execs inside that have earned bonuses consistently, over years, for awesome performance, right? You’ve rewarded success. And now you’re going to say, “I’m going to take this guy that kind of failed. And, like, you work for them.” Like, that’s hard to do inside a big company.

      Alex: It’s very hard. But I mean, in some cases, you just want to do it early. I mean, I think it actually — where it works best is where you say, “We need this product, we need this product to exist. We don’t have it right now, we haven’t spent eight years trying.” Rather than saying, “Let’s go assemble a team, and I’m going to rely on something that’s just not in our core DNA. Here’s how we’re going to go shopping. We’re not going to go shopping and value this.” And again, this is not a self-serving comment, because if somebody buys one of our failing companies for $10 million, and we have a billion-dollar fund, it doesn’t matter, right? Like, we want the companies that actually beat the incumbents, but the incumbents — the way that they could actually do great is to adopt more of this Facebook mentality. 

      And, like, the key thing is that many of these acquisitions, these kind of acqui-hire — that’s the portmanteau of acquire and hire — these acqui-hire acquisitions that Facebook made, these people now run big swaths of Facebook. So, I agree, it’s hard to do if you already have a leader in place. In that case, it just requires a very strong-willed leadership team, and an actual overt strategy that this is what we do. It becomes easier if it’s like, “Okay, we’re trying to do this new thing. Rather than assemble our own team, and they don’t know what they’re doing but they’re well-intentioned, let’s go buy a company. But let’s buy a company that hasn’t already done the thing — but a company that tried and failed to do the thing, but we’re pretty sure that these are the best triers and failers in the business.” That’s the hard thing to really measure, because most people are used to measuring outcomes and not process.

      Frank: Exactly.

      Alex: And the key thing to make this strategy work is, you actually want to over-allocate on process — and you want to weight outcome to almost zero, because you’re buying the outcomes that were, in fact, zero.

      Frank: Yep. The market is about to interview Annie Duke — “Thinking in Bets” — and this is, sort of, the essential “Thinking in Bets” notion. Which is, don’t confuse a bad outcome with, sort of, a bad bet, right?

      Alex: Right. Right. Exactly.

      Frank: Awesome. Well, thank you so much, Alex, for coming in and sharing your thoughts. For those of you in YouTube land, please like and subscribe. And for the comments thread on this, I’d love to get your input on what you thought of Alex’s idea — that what you really should do is not go after more customers, but instead go after only the best customers. So, what are examples that you’ve been trying in your own startup, where you’re trying to implement that idea? So, see you next time, go ahead and subscribe to the channel if you like it, and see you next episode.

      • Alex Rampell is a general partner at a16z where he invests in financial services and real estate companies. Prior to joining the firm, Alex was a serial entrepreneur and angel investor.

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      A Podcast About Podcasting

      Nick Quah, Connie Chan, and Sonal Chokshi

      It’s a podcast about podcasting! About the state of the industry, that is. Because a lot has changed since we recorded “a podcast about podcasts” about four years ago: podcasts, and interest in podcasting — listening, making, building — is growing. But by how much, exactly? (since various stats are constantly floating around and often out of context); and what do we even know (given that no one really knows what a download is)?

      And in fact, how do we define “podcasts”: Should the definition include audio books… why not music, too, then? So much of the podcasting ecosystem — from editing tools to the notion of a “CD phase” to music companies like Spotify doing more audio deals — stems from the legacy of the music industry. But other analogies — like that of the web and of blogging! — may be more useful for understanding the podcasting ecosystem, too. Heck, we even throw in an analogy of container ships (yes, the ocean kind!) to help out there.

      If we really think medium-native — and borrow from other mediums and entertainment models, like TV and streaming and even terrestrial radio — what may or may not apply to podcasting as experiments evolve? In this hallway-style jam of an episode, Nick Quah (writer and publisher of Hot Pod) joins a16z general partner Connie Chan (who covers consumer startups among other things) in conversation with Sonal Chokshi (who is also showrunner of the a16z Podcast) to talk about all this and more. We also discuss the obvious and the not-so-obvious aspects of monetization, discovery, search, platforms… and where are we in the cycles of industry fragmentation vs. consolidation, bundling vs. unbundling, more? And where might opportunities for entrepreneurs, toolmakers, and creators lie?

      Show Notes

      • Defining what a podcast is [2:04] and why audio has become so popular [6:45]
      • Key statistics and getting data on usage [8:24]
      • Issues with monetization [12:34] and the logistics of RSS feeds [17:55]
      • Seasonality and binge-listening [20:32]
      • Further discussion around analytics and monetization [28:38]
      • The pros and cons of interstitials [39:35]
      • Competition in podcasts, the rise of platforms, and centralization [46:38]
      • Terrestrial radio and why the audio world needs to fragment [59:21]
      • Advice for starting a podcast [1:04:16]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. So, I’m super-duper excited today, even way more than usual, because this episode is all about podcasting. For newer listeners, we actually did an episode called “A Podcast About Podcasts” about 4 years ago, which you can find on our website, a16z.com. But today we’re focusing this podcast about podcasting, since the podcasting ecosystem has evolved and changed quite a bit since then. By the way, I had hoped that Roman Mars, who was on that episode, would join us again, but he lost his voice so couldn’t.

      Our special guest today is Nick Quah, who writes “Hot Pod,” a newsletter that I’ve been following since very early on and has grown to be a go-to source all about the podcasting industry, with analysis, insights, and more. He also publishes and contributes to “Vulture” on similar topics. Also joining us for this episode is a16z general partner Connie Chan who covers consumer, the future of media, and Gen Z social, as well as trends from China, and has observed the podcasting phenomenon there and shares ideas on what more platforms can do here. And the three of us do a hallway-style jam, taking a longer pulse check on where we are right now in the podcasting industry.

      Speaking of, since we do mention some companies, please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. For more details, please also see a16z.com/disclosures.

      So we began with the latest stats on the industry, touching on structural factors and more, for about the first 15 minutes. Then we do a bunch of lightning-round style takes on how other content and entertainment models may or may not apply to podcasting for about the next 30 minutes. And finally, we go into monetization platforms, analytics, and more — which we also touch on throughout the episode — including impacts on creators. And we end on recent news and moves in the space, such as Spotify Gimlet, how to think about terrestrial radio, and more. But we began by defining a podcast, which seems obvious but isn’t, and is a rather existential question. So, guys, what is a podcast?

      What is a podcast?

      Nick: So, I mean, the real interesting thing here is, we’re in the midst of a really interesting moment of change, and there is internal conflict within the podcast community about that question. So, historically it’s been largely tethered to the notion of the RSS feed. It’s basically an audio file, or a medium of distribution, that largely happens through, you know, the technology that was carried over from blogging. And now, with the entrance of Spotify, and Pandora stepping up, and Google beginning to do whatever they’re going to do on the search engine side…

      Sonal: And Apple, already, as an entrenched player as well.

      Nick: Yeah, absolutely. iHeartMedia. And Luminary just announced their, sort of, big $100 million fundraise — and the fact that they’re going to launch in July — a couple days ago.

      Connie: With a lot of exclusive content, right? So how does, like, exclusive podcasts fit in with the old definition?

      Nick: You know, especially with the Luminary announcement, there was, like, a strong pushback from parts of the community that has been around for a while and, generally, folks who really believe in the open ecosystem. And so we have a situation in which, like, you know, the technical definition is not the popular definition anymore. And if we go from the perspective of what the ordinary consumer thinks of a podcast, that is — it becomes a cultural question, not a technical question.

      Sonal: Which, by the way, I want to say, parallels the history of the web. Because this, to me, reminds me, very much, of early blogging…

      Nick: Absolutely.

      Sonal: …and debates about what is a blog, what is an article, what is a website?

      Connie: Yeah.

      Sonal: And there was this almost religious, existential debate between the early, kind of — in fact, some of the same people because Dave Winer, one of the people who invented…

      Nick: Who also was important to the development of podcasting.

      Sonal: Right.

      Nick: It’s the same figure, yeah.

      Sonal: Exactly. But I think he was technically the first person to do a podcast, like in 2003 or something…

      Nick: Right.

      Sonal: …or one of the early people. And he’s also who specified the RSS feed, which drives the pipes, and plumbing, and ecosystem of podcasting.

      Connie: But today users don’t even think of podcasts that way. It’s like, if it’s just recorded audio of people talking, oftentimes we’ll just call that a podcast.

      Sonal: Yeah. One of my favorite things is when the people always call our videos podcasts. Like very few people find that…

      Nick: I’m mean, that’s a holdover. Like Joe Rogan still does that. There’s a lot of people who still — dual video and audio, and still call it podcasts. I mean, the way I see it is that the tension has always been between people who see podcasting as the future of blogging, and people who see podcasting as the future of radio.

      Sonal: Yes, exactly. You nailed it.

      Nick: And we see that tension clash many, many times. And I think we’re in a place where that no longer matters because, ultimately, the mass consumer will lead us where they want to go.

      Sonal: Yes. And like the web, the analogy that I would draw is to the advent of the graphical user interface, and how browsing, computing, etc. — there’s always a phase in every technology where there’s a GUI phase, where once you have an interface that’s user-friendly and easy to navigate.

      Connie: Right.

      Sonal: And what’s interesting about this is that we’re in the phase where the listening has become easy to navigate.

      Connie: And more accessible.

      Sonal: More accessible.

      Connie: Through various kinds of hardware, too. For example, listening to podcasts on their drive to work, because the cars are enabled with podcasts.

      Sonal: Right, like the smartphone-connected car, essentially.

      Connie: Or AirPods making it so easy to listen to something while multitasking.

      Sonal: And in that sense, podcasts are different than audio books, obviously, just for the sake of definition.

      Connie: But I would say, like, you can argue that, over time, that even that definition may blur.

      Sonal: Of audio books and podcasts?

      Connie: Right.

      Sonal: Yeah.

      Connie: Like, one day a podcast might just be thought of as, like, a self-published audio book.

      Nick: I have long believed that audiobooks should be central to the conversation as well, especially a couple of years ago when Audible built, sort of, an original programming team that took after podcast-style programming. And the fact of the matter is — it’s like, these are all distributors and platforms of the same kind of good. It’s just that we think of them and we class them differently. And they also, sort of, are products of different economic systems.

      Sonal: I do want to add to this mix, though, that I would not confuse music into this. And the reason is, first of all, from a creator perspective — every tool, until now, has been very music-creator centric for podcast editing, creation, etc. And so, there’s a really bad structural legacy effect of equating podcasting — I mean, we’re essentially bootstrapping tools tailored for music for podcasting, so the new wave of podcast native tools is really important. Full disclosure, we’re investors in Descript. And it democratizes the editing of podcasting because you can essentially edit audio like a Word doc. But the main point here is that I do think music should be treated very differently than podcasting.

      Connie: I completely agree.

      Sonal: Yeah.

      Connie: To me, like, it’s audio with spoken word.

      Sonal: Yep. Versus sung.

      Connie: Yeah.

      Sonal: So, I guess we’re agreeing on, just to recap the definition of podcasting — it is audio. It could potentially blur into including books. If not in a content perspective, then — to Nick’s point — then even in a distribution and business model perspective. But we agree that music should be treated differently.

      Nick: Absolutely.

      Sonal: And the common denominator here is spoken word.

      Growing popularity of audio

      Nick: The Infinite Dial Study, which is, sort of, an annual survey conducted by Edison Research — they just announced their latest results earlier this afternoon. The most interesting thing is that there were increases in both audio books and podcasting. So podcasting had significantly, like, a large leap this year. But on audio books, like, after a couple of years of largely being flat, it’s been increased again. And I think that’s a, sort of, really interesting question because I can’t quite think of a structural reason why that would be the case other than…

      Connie: AirPods.

      Nick: …it’s the, sort of, like tethered effect.

      Connie: In addition to that, you have all kinds of really easy-to-set-up wireless speakers at home that also make it [easier] to…

      Sonal: Yeah, like Alexa.

      Connie: Yeah. To consume this kind of content.

      Nick: It reminds me of, like, what people say about the Kindle and romance novels. It helped, like, sales increase because it made people, like, more willing to buy it and consume it, because then nobody would judge them.

      Sonal: Oh, the judgment side. Interesting. For me, it’s actually ease of access, because I used to be — I’m really embarrassed to admit this publicly — I used to subscribe to the Harlequin romance on demand service, where you’d get, like, the books a month, and you’d pay, like, $11 or — I can’t remember what it was. Because I’ve always been a huge reader of romance novels as a very nice, lightweight thing to do. But what’s the analogy to podcasting? What’s the connection?

      Nick: To me, I think it’s more ease of access around better hardware.

      Sonal: On demand, get it quickly. So speaking of the data — and you mentioned that the Edison Research study came out today. And that’s, sort of, the definitive and longest running survey of digital media consumer behavior — in America, at least. But I hear a lot of mixed messages. I see, like, people cite this stat and that stat out of context. So, why don’t we just do a quick pulse check on what are the key stats. And Nick, maybe you could recap for us what the key stats or big trends to know are here.

      Nick: Sort of, I think there are a couple of big takeaways here. One is, when it comes to the familiarity of the notion of podcasting — and this doesn’t mean people who heard the word actually know what it is — it’s officially hit 70% of all Americans. And when it comes to the number of people who’ve actually tried out podcasting — you know, maybe they didn’t stick around a bit but they just tried it, at least — it’s gone over 50%, so about — an estimated 144 million Americans. Retention rates are, sort of, like, really interesting. Like, monthly podcast listening is also going up. It’s now 32% of Americans, up 26% from last year. That’s a pretty big leap.

      Sonal: I mean, just, that’s one third. That’s a lot.

      Nick: Yeah. And there’s also a really interesting slide in here attributing some of the increase to Spotify. There is a stat here that shows, among Spotify listeners between the ages of 12 to 24, monthly podcast listening went up to 53%. And so, there’s a lot going on. I think, currently, it’s such a moment of flux. It’s a little unclear what the structural pillars are anymore. And I think this is one of those things where we’re just going to have to, like, look back at this moment to figure out where we turn.

      Sonal: So, what’s a high-level recap on that summary of the stats?

      Nick: The high-level is that this past year has seen an unprecedented growth. For the longest time, podcast growth has been steadily and slow, and now it feels like it’s taking some sort of a leap. And so, I feel like this past year has been the moment where it’s tipped into some form of mainstream.

      Sonal: That’s fantastic. So, potentially, an “inflection point” as people like to say in the business.

      Connie: The usage of podcasts and the consumption of it has risen dramatically in the last year or two. But what always shocks me is that the revenue that podcasts generate is still such a small amount, given how many hours people are spending consuming this kind of content.

      Nick: So, there is a study out there from the IAB — the caveat being, it was funded and financed by a constellation of podcast companies — that puts the number at around 600 million-plus-plus this past — last year. And it’s projected to keep growing, of course. Monetization is a severe issue. And it largely has to do with the fact that podcasting, as a technology, hasn’t quite caught up to how the rest of the internet, kind of, works in terms of dynamic ad insertion. And it doesn’t allow, like, heavy increases in inventory and swap outs in inventory, in a way that a lot of advertisers are now accustomed to getting from, you know, marketplaces like Facebook.

      Connie: And then, even that, like, from an advertiser’s standpoint, you’re paying per download, because you aren’t getting, like, these per-listen metrics back. So from the advertising standpoint, it’s still really hard for them to measure the ROI from sponsoring a podcast.

      Sonal: Yeah.

      Nick: Yeah. And that’s why, historically, we’ve seen a bunch of the activity among advertising from direct-response advertisers, because they have a secondary metric of conversions on their promo codes and whatnot. And what they’re able to find is that the conversion rates are good. But when it comes to something like a brand advertiser, or an advertiser that needs to, you know, lay an impression on a consumer over a 5-, 10-year period, they need to know that they’re hitting the people that they’re hitting.

      There are a lot of movements right now towards standardizing what even a listen means. And this will become increasingly complicated as Spotify and Pandora…

      Sonal: Everywhere.

      Nick: Right.

      Sonal: I mean, right now, you don’t know, is it a download, is it a click, is it open, is it <inaudible>? I mean, who the fuck knows? It’s, like, a mess.

      Connie: Or like how long did you listen to it, right?

      Sonal: Right, the engagement. So that’s actually what I care the most about as a creator. Because when I was at WIRED, Chartbeat changed me as an editor. And I need to know where people drop off. That is a number one thing. So I don’t know if you even know this, Nick. We were in the launch set for when Spotify launched their first move into podcasting, in 2015. They selected us as one of their media outlets, because our podcast was one of the very few that covered tech in a thoughtful way. And the reason I was so excited about Spotify — because Spotify didn’t really have much of a podcasting audience back then…

      Connie: Yeah.

      Sonal: …was they showed me this really beautiful dashboard that showed you the potential, and where people drop off.

      Connie: But you don’t get that from all the other places…

      Sonal: No, you don’t.

      Connie: …our podcasts are distributed.

      Sonal: It’s still limited because not all of our listeners are listening on Spotify.

      Connie: Right, right.

      Sonal: They’re on SoundCloud, they’re on iTunes.

      Connie: Right.

      Sonal: They’re in a bunch of different apps. And iTunes, by the way, also announced this — I think, what, last year? James Boggs announced that you can actually have drop-off…

      Nick: Yeah. They rolled out more granular in-episode analytics.

      Issues with monetization

      Connie: Another thing I would push back on those — like, I don’t actually think advertisements are the only way you can monetize podcasts.

      Sonal: Yes, I agree wholeheartedly.

      Connie: I feel really, really strongly about that. Because even as someone who consumes podcasts, ads are extremely annoying to listen to. And this is where I look at other business models that are working in Asia for podcasts that I think could, very much, translate here.

      Nick: Yeah. So a couple of points on that. And so, a situation which — there are behaviors in internet usage, in gaming, in media consumption — in China, Japan, and Korea — Australia, Malaysia, Singapore — that doesn’t occur here. Maybe through path dependency reasons, maybe through, sort of, technical habituation reasons. And, yes, so we’ve already seen, like, a really healthy growth of the number of podcasts using Patreon as — maybe not a primary, but a strong supplementary business model. “Chapo Trap House” is an example of this. There are a bunch of podcasts collectives that rely on Patreon for this. And there’s also, like, Slate Plus being a, sort of, a central model to Slate as a digital media publisher that also heavily indexes on podcasting.

      But, you know, I think I’ve always found this “lack of data” conversation a little interesting, because whether or not advertisers feel confident in the measurement, and what the data is, sort of, trying to reflect in terms of reality, the world continues to spin, and, like, people do end up paying — like, converting as a promo code. And so there is a strong sense that podcasting is a very powerful driver of consumers. And it’s a powerful advertising driver <Oh, yeah!> even though we’re not able to tell specifically how many people that get hit in terms of just the analytics of it. And so, there’s this fear, I think, among a lot of people that, you know, the analytics side will end up driving way too much of the conversation — and ends up dictating the behavior of creators and publishers in a way that might end up being, you know, unhealthy or counter-intuitive to the relationship between the listener and the creator.

      Connie: The problem with that, I think, is like — yes, analytics may skew what kinds of content they put out and how they engage with their audience. But, like, really, analytics is just a nicer way of saying revenue. Because at the end of the day, your analytics are a reflection of how many listeners you’re getting, right? And this is where I think, like…

      Sonal: I don’t agree actually completely. I agree with you from a business perspective. But as a creator, the analytics tell me about community. And one of my favorite talks on the early days of resurgence of podcasting was — Marco Arment gave a talk. I was at XOXO in 2013. And it was basically about the resurgence of podcasting — the early signaling of that — and podcasts as a movement. Because what’s really unique for the first time, when you think about the first wave of podcasting with all the indie bloggers, we now have brands podcasting. And sometimes they’re not actually looking for direct revenue through that, it’s a way to really connect intimately with your audience. I mean, it’s essentially a movement brought live in audio form.

      Connie: Okay, fair. So, I mean, there are types of content where it’s not about monetization. But for a lot of creators, I do think revenue is one kind of proxy for…

      Sonal: Absolutely.

      Connie: …how much value they’re providing their listeners. And I also think that, like, we’re in such, such baby phases of how podcasters should be able to monetize. Like, honestly, they shouldn’t be having to ask their listeners to go to other sites to pay them, like, a monthly fee.

      Sonal: Oh, yeah. You can’t do it in-app.

      Connie: I mean, this is where the platforms are going to start rolling out subscriptions. I think some are going to roll out, like, other ways of paying for packages or bundles of content. And I think that’s when you’re going to see creators really unleash, like, much better content, where they don’t have to focus on mainstream audiences. But they might focus on smaller audiences that are willing to pay for that.

      Nick: So, actually, I’m like, really fascinated in terms of the concept. If analytics is being the, sort of, like, proxy for revenue here, it’s strange because I’ve always, sort of, viewed analytics as, you know, a certain kind of representation of reality. And it just so happens that advertisers, at this point in time, are really reliant on a certain expectation of a kind of analytics, in order to discern whether a media product is effective in a way that they want it to be. And there’s this larger conversation about platforms in general. You know, switching metrics, or tweaking metrics or, you know, in some cases, ballooning them in order to control and manage that narrative and relationship with the advertiser.

      Connie: No, I completely agree, analytics matters for an advertising model. But what I’m saying is, like, the advertising model is actually not a good model to monetize podcasts.

      Nick: No, that, we completely agree with. But it’s a situation in which, like, it is the revenue that a lot of people — a lot of publishers and creators feel most comfortable with because that’s all they know right now. And, we, kind of…

      Sonal: I think it’s actually also a legacy. This is where I think we need to think, again, very native in a new medium. This is where we do ourselves a huge disservice. Like, the early days of the web when media outlets would put, like, a fricking, you know…

      Nick: Banner ad. Yeah…

      Sonal: …home page analog on their website. Right. Exactly. Like, we need to think very natively in this medium. And we have a huge opportunity for the first time because we have such an intimacy, a slipperiness, a connection with podcasting that’s visceral. That’s — I mean, personally, I think it’s unlike any other medium I’ve ever seen. I feel like I’ve found my voice on this medium quite honestly, but — so, I do think that we have an opportunity here, because we’re so stuck on the legacy of — and, in fact, this goes back to something we started with, which is what is the definition of a podcast?

      So, I think that the thing to revisit here is that — the underlying pipes and infrastructure. And I know people don’t expect this when we’re talking about an episode about podcasts, but I think it’s really important because it informs this conversation. It is RSS feeds. It is literally an ecosystem of pipes that are connected by feeds, talking to feeds, talking to feeds. This is both a structural, huge limitation causing major fragmentation in the industry, major limitations on what’s possible, with what creators can do to even connect the dots — because the unit of analysis is limited to what you can actually send in a feed.

      Connie: Right.

      Sonal: And that has certain trade-offs to it. And this actually reminds me of container ships — like, physical, large shipping ships — like Maersk, etc., that you see in the ocean. And one of the novel things about container ships is about what they did to creating trade across the world. And because they’re multimodal — they go from airplane to ship, to truck, to yard — they allowed so much collaboration and connection around the world. That’s what feeds are doing for the podcast ecosystem. What’s missing, however, is — just like a container ship — containers are rectangular boxes that are very limited in what you can actually fit into them. And people, therefore, need to fit the shape of their goods to fit in those boxes. And the entire ecosystem for physical container ships is architected around being able to lift things out and in.

      That is the same thing that’s happening in podcasting right now. The containers are connecting all of us in this feed ecosystem, but they’re also dictating what information travels where, and in what form. And I just want to point this out — no matter how wonky it seems — because that structure dictates so much of what the current batch of tools can and can’t do when it comes to analytics, the discovery, and more — all across the board. And it’s where platforms and tool builders have a huge opportunity to cleverly address, or even bypass, those containers once we get past this phase of where the podcasting industry is structurally right now.

      Connie: Yeah. I just think, like, we are in such early, early, early innings of what podcasts can be.

      Sonal: Yes.

      Connie: Because if you think about it. Again, this is not using the technical definition of a podcast, but using this cultural definition of, like, audio recorded content, right? Most of the time you’re consuming that kind of content on an internet-enabled device. It’s not like you’re downloading it onto your computer and then, like, using a USB stick to transfer it to your phone, right?

      Sonal: Mm-hmm.

      Connie: And so therefore, like, we are not monetizing this stuff, or even creating features on top of it that are internet native. There’s just so much stuff we’re not even tapping into. And it’s such a shame, because we’re consuming these things on internet-enabled devices, and yet we’re using the same business model as televisions…

      Sonal: Where you can’t even do anything.

      Connie: …which is not meant to be interactive.

      Sonal: Yeah.

      Connie: And there’s, like, right now, very little interaction with the podcast which I think is such a shame.

      Seasonality and bingeing

      Sonal: So I want to ask you guys, kind of, lightning-round style on a couple of neat things that are artifacts of the existing world of content, and how we think they’re going to play out with podcasting. So let’s just — I’m gonna throw out a phrase…

      Connie: And I think we should get your take too…

      Sonal: I will.

      Connie: …because you have more expertise on podcasts than anyone in this office.

      Sonal: You’re right. I forget to do that as the host sometimes. Okay. So, I want to ask you guys about seasonality. Like, what do you guys think of this trend of people dropping podcast seasons?

      Nick: So, I love seasonality. Like, it gives me a feeling of momentum. And also, we’re currently living in a moment where there’s all things happening all the time, so many things to consume — I would like things to have definite ends. And I’m a big fan of seasonality personally.

      Sonal: Yeah.

      Connie: I think it also makes it easier for bundling…

      Nick: Yes.

      Connie: …and different pricing down the line.

      Sonal: Oh, fascinating. So for me, seasonality is — so, when I think of the long tail of content — and Chris Anderson wrote the fundamental piece and book on this — it’s this idea of an infinite shelf space. And to me, things being in software and being digital, it’s unbounded to the point of being pointlessly infinite. And forcing a false scarcity is my favorite thing that, like, box-in-a-month companies do — like Stitch Fix and makeup, whatever. It’s a way of curating and creating scarcity in a world of abundance. And I think that’s a really interesting packaging thing for any kind of content across the board. And especially for podcasting. Because there is no — you’re essentially in an infinite scroll in the audible world. You don’t know where you are, you have no context, you’re not plugged into a specific thing, because you’re living in this weird ecosystem of voice and show — or episode depending on how you’re listening. So that’s my quick take on seasonality.

      Connie: Okay.

      Nick: Love it.

      Sonal: Okay. So binge-watching. This is related to seasonality. One of the most fascinating things about [the] Netflix phenomenon in the space of visual content is, they realized, like, “Wait a minute, we don’t have to do weekly things, we can drop everything at once. Not release it as a season that spreads out once a week or whatever the pace is, and allow binge-watching.”

      Connie: I think binge-watching is great, and it’s natural human behavior for any kind of content. I suffer from it myself. Like, I was the kind of person — I would watch the series “24” — I would watch a season in, like, 30 hours.

      Sonal: I did that too with “Stranger Things” and everything.

      Connie: Yeah, yeah. And it’s just natural human behavior, and so I think it’s great.

      Sonal: That we want to just be addicted and go deep all at once, and we can’t stop ourselves?

      Connie: And, actually, in terms of — for the creator, I think it’s a good thing, because you don’t want that listener to, kind of, forget about it.

      Sonal: Yep.

      Nick: I binge-watch all the time so I’m just going to take, devil’s advocate, that I only, like, believe about 80% of. One is, I actually think that binge-watching or binge-shopping has actually caused attention to a given show to deteriorate, right? It used to be the case where, when a TV show drops weekly, there was, sort of, a positive conversation that is drawn out over a longer period of time if that show has hit. I thought about…

      Sonal: You mean, like, the water-cooler conversation?

      Nick: Absolutely. Like, “True Detective,” “Game of Thrones,” basically, everything that HBO — like that sort of structure of it, I really liked that water-cooler conversation. And I like to be on the same, sort of, page as other people when I’m having that conversation. And that’s something I’ve never gotten with a binge show. I loved the “Russian Doll.” I can’t find a single person to talk to about it who, you know, follows it around the same time. And like, I can guarantee, in about a month, I’m going to forget about that show.

      To use a tortured metaphor, the thing about binge TV that I enjoy really doing, but I feel a little bit sick of doing it afterwards — it reminds me of, like, you know, that thing when, like, parents say that they’d do to certain kids where — if they catch that kid smoking one cigarette, they’d make that kid smoke the entire packet in one sitting?

      Sonal: Oh, yeah.

      Nick: That’s, kind of, how I feel after when I binge a season. I feel like I don’t want to watch TV for, like, a month.

      Connie: But it’s, like, inevitable, you know? I feel like this is a behavior you can’t stop.

      Sonal: So my whole thesis about this, which is similar to screen time and kids — because people always have these stupid religious debates over it — it’s not so much the act of doing it or not doing it, it’s why you do it. So, if you’re someone who is binge-watching because you’re depressed, that’s not good. But if you’re someone who is binge-watching because you just can’t stop watching the show, that’s great.

      I will say, to push back on your point, Nick — because I know you’re taking the devil’s advocate — but I think that what you’re describing, this problem of the, like, water-cooler thing that, Connie, you’d labeled — it’s actually an artifact of technology not quite being there. Because there is a movement of second-screen technologies that are allowing more — there’s forums online, like Reddit, that aggregate.

      To give you a perfect example of this. When I finished “The Three-Body Problem,” the first thing I did was go trawl the web to find all the forums and all the people talking about it, so I could find my people and talk about it, and find other people who loved it. And so there are tools that are emerging that allow conversations, to then, to your point — the water-cooler — to be aggregated asynchronously. And there will be, I think, a second-screen phenomenon happening with pod listening and binge-listening, as we start having the technology ecosystem grow.

      Connie: I can see how, you know, you don’t want to spoil the ending. So you won’t actually go to that forum until you finish your book.

      Sonal: You’re absolutely right. And, actually, I like that you can have a choice. Because in spoiler-alert culture, which Nick is slightly hinting that he misses, at least on the devil’s advocate mode…

      Nick: I do.

      Sonal: …there is sort of like a thing where you can actually choose to check out of things, luckily, so you’re not, like, stuck in a room with everyone talking. And then you are screwed, because you missed, like, the closing season of “Dallas” or whatever show it was.

      The other point I want to make about binge-listening, in this context is — with binge-watching, new types of narratives are happening, I’m very curious about what will happen — as we start seeing binge-listening of podcast seasons, or podcast episodes — to narrative, and how that’s going to change that category of podcasts — where, would a “Serial” change the way it tells stories because people are bingeing it?

      Connie: Well, then it becomes an audio book.

      Sonal: Oh, interesting.

      Connie: Right?

      Sonal: Then it becomes an audio book. Oh my God, I would have argued it to almost the opposite item in the spectrum. Because it’s, sort of, going through a book very quickly.

      But the flip side of it is — when I’m thinking the analog with binge-watching — is that you can watch an entire season and it changes the way — you don’t have to have a cliffhanger at the end of every episode. Whereas, even in a chapter, people still have a little bit of these things.

      Connie: Oh, I see. I see.

      Sonal: Right, narrative.

      Connie: I will say, I think “Serial” would have made a lot more money if it allowed people to pay. I think, on the margin, binge-listening helps creators. Because if you can make someone pay for, like, a whole season at once, and maybe give them like one or two episodes for free, it’s better than hoping that they’re going to come back every week, right?

      Nick: The “Serial” example is actually really, really interesting. “Serial” itself was an innovation of the form, because it stuck to what podcasting was able to do at that time. Prior to the existence of “Serial,” it was incredibly difficult to tell a serialized story over the radio in the form that they did it. And secondarily to that, they told that story in semi-real time. And that’s something that they, sort of, looked at the structure of what the distribution format was and they go, “We’ll go and try that out, we’ll see what happens.” And so this is a little bit of, like, them playing perfectly to the form there.

      And I want to, sort of, go back a little bit to the point about, like, the second-screen experience and the, sort of, the death of the water-cooler. So, I love second-screen experiences. I live for NBA Twitter, I live for Bachelor Twitter. But I’ve got to say, I do like that experience with physical people, and that I miss hanging out and watching TV with my friends sometimes at the same pace. That’s all I got.

      Connie: I just think, like, ever since DVR arrived, like, we kind of lost it already.

      Sonal: I think you guys are both being very falsely nostalgic for a past that never was. Because I actually think — I mean, yes, there’s a reality to be physically present, but again, we’re in the early innings with all of this. We’re investors in a company called Bigscreen, where you can essentially share in this ambient intimacy — like, hang out in VR. Like, when there is a digital overlay over the physical world. Just like people connect on Twitter for ambient intimacy — the cocktail party of the web — there’ll be a physical, like, experience, that you have — similar level of satisfaction in hanging out in real time with your friends. And it’s just an artifact of technology that we’re not 100% there yet. That’s what I would argue, at least.

      But back to the binge-watching thing. I was going to add that when a season drops all at once, I add it to my playlist, but I never watch it. Because what’s also missing in this space — and this is, again, why I love the idea of binge-watching/listening for podcasting — is the concept of virality. Viral hits don’t happen instantly unless you’re, like, a Joe Rogan Experience, an Elon Musk smoking pot on air.

      Connie: Yeah.

      Sonal: Like, it’s, sort of — or a cult of personality show. It’s slow-burn type of virality. And so seeing what people are talking about and what resonates is hugely important for creators. Not because you freaking want to crowdsource what you want to say, but you do want to know it doesn’t go in a black hole.

      The need for better analytics

      Connie: I would love a world where, in the future, you’ll know which parts of the podcast the audience…

      Sonal: Resonated.

      Connie: …liked the most.

      Sonal: Right. My proxy for that, by the way, is — I do Twitter searches all the time for the commentary, so it’s a very skewed sample, but it’s helpful and I push the editors to do this — to close this loop, even, if they’re not active on Twitter, because there is no other way to see what resonated. And as soon…

      Connie: But can’t…

      Sonal: Yeah.

      Connie: But can’t you see, like, a platform just like saying, “Tap your screen if you like this part?”

      Sonal: Oh, totally. Well, I don’t know if this is public. Do you know this, Nick? But <redacted> is doing screenshot — audio shots of podcasting?

      Nick: Yeah, I’ve heard about this. Yeah.

      Sonal: Yeah. Is it public, do you know?

      Nick: Not yet.

      Sonal: Okay. But there will be podcasts screenshotting, and audio clips. I’m curious to see, with or without the transcript, Connie — to your point, about the importance of that — whether those will go viral.

      Connie: It’s crazy to me that these things don’t have automatic transcription on the top hits. Like, that’s such an easy technical thing to do. And for a listener, that would mean that I don’t have to just pause and say like, “Oh, yes. Remember, like, go back to the 1 minute 30 seconds mark later on and take notes.”

      Sonal: Well, I actually love that, too, because one of the biggest limitations of podcasting is the lack of a “screenshot equivalent.”

      Connie: But that exists in China already. Not only can I see the transcript, I can then comment on it. And I can make it so only my friends can see it, or I can make it so the entire public can see it. And then there’s a discourse…

      Sonal: That’s amazing. Right now, we have to manually upload transcripts.

      Connie: And you basically have threaded conversations around parts of your podcasts. And so it’s okay if the listener doesn’t even get to the end, because you can have a highlights feed — all kinds of stuff right now that we just are not doing. And so I think this is, like, where the platforms can get much better at creating. Like, even if they, like, just chunked up the best clips, right? Or maybe you, as the creator — you can, like, throw out which clips you think are the best. Make it easy for them to repost on other social mediums, or make as, like, background music to whatever. And I think…

      Sonal: Yeah. You can do that, actually, now on some of these tools. But to your point, it’s fragmented, it’s not centralized on a single user experience.

      Connie: Fragmented. And I think, like, the main platforms don’t allow that, right?

      Sonal: Mm-hmm.

      Connie: And so…

      Sonal: Currently, no. Spotify, and iTunes, and others don’t. In fact, this is, again, where the ecosystem is so fragmented, because the side players — there’s a whole budding ecosystem of tools that are doing this kind of thing.

      Connie: So, again, it goes back to — you know, like likes, and comments, and payments, and tips — that’s just like a form of showing how much you like something. Creators don’t know which pieces of their podcast were the best parts of the episode. They don’t know where the highlights are.

      Sonal: They don’t know any of it. It’s a black hole. But on the metrics, I do want to say that one of my favorite analytics for podcast success — because I do think that we need to think about what you’re measuring for, for the type of show you are — and in our case, what I care about as editor for the show is insights per minute. And this is the same thing…

      Connie: Cool.

      Sonal: …as insights per inch, in terms of like going down a verbal post.

      Connie: Yeah.

      Sonal: Because when you have a brand collective and not a cult-of-personality driven show — this is, again, where the metrics for the type of show need to vary as well, in my view. For our kind of show, if you’re not, like, a famous personality, then the insights per minute matter a lot to get people to stick and stay. And then secondly, when you think of audience discovery — audience and movements of people and fans aggregating around a piece of content — then I care about — if a show has, say, a drop-off halfway, as a drop-off point, if the first half are people who are mainstream interested in learning about quantum computing, and then they drop off 50%, I consider that a huge metric of success. And if the remaining 50% that stick around — a much smaller subset of people who are developers in quantum computing, are interested in building quantum computing, are physicists — then that’s a huge metric of success. So for me, again, this is, again, another granular way of thinking about the type of show, the type of content, etc.

      Now we can’t do any of this right now. But as we introduce new storytelling and forms in podcasting, I think we’ll be thinking a lot more differently than the obvious on those fronts, too, and about podcast engagement. Which, by the way, one quick factoid for you guys — the number one thing I hear from all of the publisher network. Because one of the things that I did when I came here was reach out to various people to beg them to put their authors on the podcast — this was before authors became — like, going on podcasts became the thing to do.

      Connie: Yeah.

      Sonal: And there’s nothing that moves books the way podcasts do. I’ve heard this over, and over, and over again from all of my publishing industry friends.

      Nick: I heard the exact same thing. The way that the podcast experience is currently constructed, it drives sales. But the question is — is that when other platforms — or when the experience changes due to technical innovations or new features added, would it fundamentally change that relationship? Will there be the same kind of sales push that we experience right now? It’s an open question.

      Sonal: I do think it’s an open question.

      Connie: I think it could totally work. I mean, like, to me it’s like the same way QVC is a great way to sell stuff, like, podcasts is a great way to sell content — written content that people want to read. But I think this is a bigger problem with the book publishing industry, meaning that they’re not selling books in an internet native way.

      Sonal: Right.

      Connie: There’s no great way to figure out the highlights of a book, there’s no way for me to read the first chapter for free, there’s no way for me to, like, get a sense of, “Do I want to pay for this entire book?”

      Nick: I do that all at a bookstore by just skimming them. I mean, like, it’s… <laughter>

      Connie: In a physical bookstore, yes. In a physical bookstore, you can do all these things, but on Amazon, you still can’t.

      Sonal: Right. This is another way where I think we’re not thinking of the native medium. Because, it’s crazy to me that books, which are self-contained, with no context, are still decoupled in audio book form. And it’s equally crazy to me, that podcasting, because of the structural limitation of the feed pipes, don’t actually have context built into them where you can actually tie a podcast into the context of a broader show, more by this author, more on the topic — to your point about PDFs, and show notes, and related materials. It’s crazy to me that there isn’t a web-linked ecosystem for podcasting yet.

      Connie: Because none of this stuff is being sold in an internet native way. I just think, like, that right now, the way we sell books, it’s like — if you had no movie trailer and you only had the movie poster.

      Sonal: Yeah. <laughter>

      Connie: Right? Like…

      Sonal: Yeah.

      Nick: It depends on the movie poster.

      Connie: You’re, like, buying the book based off the cover and maybe some quotes by people who’ve read it, but you don’t get to even see the trailer. And this totally actually skews the creator’s incentive for what kind of content to create. So, like, for a book, like — are you going to pay $20 for like a 20-page book? Or will you feel better about paying $20 for like a 170-page book? And then authors might have to write extra words for the sake of selling a, you know…

      Sonal: Oh, that reminds me of [the] early days of Charles Dickens where he was paid by the word, and that was, like, a funny artifact of the way the monetization was happening. But I would argue on the flip side of that, on the creator side, I think it’s more important to find your community. Because the beautiful thing about — again, podcasts are movements. Groups of people following either a show, or an episode, or a topic, “Serial” fans, whatever it is. And so, when you have “1,000 true fans,” in the Kevin Kelly phrase, that are following a particular book author, or a particular topic, or a particular podcast — in our case, what we’re doing is, we’re mobilizing the fan base. Not because of that author, but because of the way that we do our take with that author. Like it’s, sort of, the a16z take on it. So when we did Yuval Harari, it was me and Kyle talking to him about all kinds of random stuff that was probably not even related to his book.

      Connie: Right.

      Sonal: The point is that it’s a way to mobilize your movement, your fan base. And this goes to Nick’s earlier point about Patreon and fan bases, or Marco Arment’s point about brand as intimate connection.

      Nick: So, my theory on this, sort of, notion of, like, what people would pay for — people will pay as much for a thing based on how valuable they think the thing is. And so, it’s equally plausible that a person looks at a 20-page book and thinks that it’s worth $20, as it is that a person looks at a 170-page book and thinks that they will pay $20 for that. It really depends on how that person — or how it’s messaged to this consumer, what value it is, right?

      Sonal: Yeah.

      Nick: And so this ties back a little bit to the notion of advertisers and analytics. Analytics, as constructed by a technology company, by a platform, by a data team — is an effort to tell the advertisers, “This is how valuable you should think this is.” And in the art world, value is constructed in a whole different amorphous way.

      Sonal: Yes.

      Nick: And so I think it’s, like, a one-to-one objectivity of — what is the right metric, or how do we find the truth of the value of a certain thing? These are socially constructed things. And so I think that should be a consideration when it comes to when we think about — even the book publishing industry. I guess, we should argue that celebrity books should be priced a lot higher than it is. But, you know, that’s just me.

      Connie: Books is just one example, though. Like, if you think about, like, a YouTube video. Like, the creator is incented to make it long enough so you don’t purchase one pre-roll ad, but also put, like, another ad in the middle. Which means the video has to be long enough to have enough gap time between the ads, right?

      Sonal: Not really, because the most popular videos on YouTube that do really well are the short, quick takes, or tutorials. Or, like, in those cases, that’s another example of — I mean, I think that’s the reason why tutorial culture has taken off because people are self-selected into, like, learning about X, Y, or Z.

      Connie: But, like, some creators will lengthen their videos so they can put in a second ad.

      Sonal: Yeah. I think those, to me, are the more old school creators that are doing that to monetize in that way. They’re not the ones who are the influencer creators, because the influencer creators have their eye on a much bigger ball game. They’re looking at moving their own freaking makeup lines, or like — you know what I mean?

      Connie: Yeah, yeah.

      Sonal: Or, like, other things. But, yes, that is sort of like the early phase of every platform and medium — is that you have a quick way to, kind of, game it to get what you need. But I don’t know if that works for the long-lasting players.

      Nick: YouTube in that situation is the arbiter of the data that tells advertisers what to value. But it’s also the arbiter of the data that tells creators how to value the way that they’re creating something. It also becomes a situation where YouTube is the thing that interprets human behavior, and makes assumptions based on those interpretations through what people are valuing. And so this is, like, YouTube, sort of, defining that reality and pulling levers in a bunch of different ways. And they may be correct, they may not be correct. In any case, it’s all a proxy of reality that may or may not be aligned. We don’t know necessarily.

      Sonal: I agree. I agree it’s socially constructed and value is created. And a lot of it is limited by the tools people have for thinking about pricing. And they have heuristics for doing that based on those structures. I would also say that there’s a really interesting opportunity, especially with podcasts, to flip the model — where fans get paid. And in fact, Kevin Kelly made this really interesting argument in his book “Inevitable” about how, when you swap your paradigm for thinking about attention in an abundant software world, which is what we’re talking about here, abundant digital world bits are infinite…

      Connie: Yep.

      Sonal: …there’s no limit on airwaves in this context, you can actually flip the model where fans can monetize their attention. So you actually reorient. And this is actually the premise of crypto, right? Or one of the premises of crypto. At least in the notion of crypto networks, where right now, the locus of data controls the platforms. With crypto, you can actually invert that, where the user is the container of the data. So if you think about this in the context of media creation and podcasting, how interesting to think about a fan monetizing their attention? Because if a fan is a sum of all the shows they watch, maybe an advertiser wants to buy that fan and the fan directly monetizes that attention. I know that sounds crazy, but I don’t think that’s impossible in a world like this. You guys are both looking at me like I’m nuts.

      Connie: I think if platforms can do that, like, there’s all the stuff they need to experiment with before they even can get to something like that.

      Sonal: Yeah, yeah. That is if you believe it has to go step wise. Because sometimes, technologies can leap. I agree with you, I think it will be incremental.

      Connie: I’m like, “If we can’t even get subscriptions or tips of, like…”

      Small bites of content

      Sonal: We can’t even get downloads, for fuck’s sake. All right, I’m going to do another quick — I want to hear you quick lightning round take on interstitials and podcasting. Any thoughts on that? The idea of, like, you know, title slides, or breaks, or segmentation, etc.

      Nick: I’m pro interstitials. It’s like, you know, it’s really important to orient your audience and to teach them how to listen to the thing. It’s an important creative tool. That’s my view on that.

      Sonal: Connie, I feel like you have a lot of thoughts on this because it feels so China native, like, what people do and…

      Connie: Describe more what you mean by interstitial.

      Sonal: I mean, more, just like, it’s kind of to your point about there being granularity. Like, you can actually break up a show into sub-parts by having little breaks or…

      Connie: Oh, yeah, yeah, yeah. I think interstitials [are] great. Because, again, it allows me to show you which parts of your episode I value the most and which ones I’m willing to pay for.

      Sonal: Yeah, for me, I will say that we tried some early experiments with segmentation. Because I got this funny feedback from people that they’re like, “I listened to the podcast on the road, and my commute is 10 minutes. I wish they were 10 minutes long.” And then someone else is like, “My commute is 20 minutes. I wish they were 20 minutes long.” And then someone else is like, “My commute is 30 minutes,” or 40 minutes. And they have this ideal time. 

      For us, at least, 30 minutes has been the sweet spot in terms of, like, the ideal podcast size. But I don’t think there’s a rule of thumb. Like, some of our most popular episodes are an hour and also 20 minutes, so I don’t know. But I did, because of that — I wanted kids on campuses, like, at Stanford or wherever, to have a way of listening to an episode and, kind of, have like a nice natural stop off point. Because when you’re watching a show, the ability to, kind of, pause — so to me, interstitials are a way of creating a little bit of those moments and breaks. But then what I realized is that as an artifact of this industry, all the tools save your spot in where you were playing last in your player.

      Connie: Yeah.

      Sonal: And so it, kind of, became a moot point. So that experiment didn’t really work. But the driver for it is this thesis that — and, you know, Dixon says, the internet is made for snacking?

      Connie: Yeah.

      Sonal: And podcasts can be beautifully long-form. But I also think that there’s a consumption mode and very short, micro waiting moments, to use a term from a Park paper on this concept — that when you’re waiting in line, can you listen to a quick bite of content? Not just watch something on your thing, not just listen to it.

      Connie: Super interesting.

      Sonal: Yeah.

      Connie: Yeah.

      Sonal: And I wonder if we can fill micro waiting moments. And so I wonder if interstitials would play an interesting role as, like, a micro waiting moment.

      Connie: To do that, I feel like you need really good discovery.

      Sonal: Oh, yeah. Or following a show, right?

      Connie: Because the likelihood of me, like, hitting something that I don’t like causes, like, this fear in the listener.

      Sonal: Of course. Unless you are then — which currently is a model — of following a show or a personality.

      Connie: Yeah, you just have to have so much trust…

      Sonal: Yes, yes.

      Connie: …that it’s going to spin up the right thing.

      Sonal: Yes. Because, right, because in the cult-of-personality model, people are following the person, not necessarily the guests.

      Nick: I’ll just say that the notion of short-form audio is one that is constantly talked about. This also, just as another reminder, like, what Anchor essentially attempted to do at the very beginning of their journey, and what Odeo tried to do. And it’s one of those things where it didn’t — both of those iterations didn’t quite work.

      Sonal: No.

      Nick: We don’t know if that has anything to do with what people want, or if it’s the case that people were not ready for that yet.

      Sonal: I would argue the last one, because we have seen over and over with technology, there was, like, five Facebooks before there was a Facebook that works.

      Nick: I subscribe to the view of the world in which human beings are generally plastic, and so you could force a human being to accept just about anything. And so, it’s a question of whether the right startup — or the right platform — executes the right experiment, or the right time, with the right group of people. That’s just, kind of, how these things work.

      Sonal: Yeah. Human beings are creators of emergent behaviors, because this is where you can never predict the second order effects of new mediums, right?

      Connie: Yeah.

      Sonal: Like, Twitter spawned all kinds of interesting emergent behaviors. And that is the fundamental truth of the evolution of all kinds of technologies.

      Connie: But it’s all technically — I mean, this is not like cutting-edge science or technology that doesn’t exist yet. It’s just the platform hasn’t put all of these things in place.

      Sonal: Yeah.

      Nick: But the fact of the matter is, is that stuff like social audio, stuff like Anchor’s initial bid to be the Twitter of audio — this stuff like Odeo, which is what Twitter was before Twitter became Twitter, which is essentially…

      Sonal: Oh, right.

      Nick: …Twitter for audio — is that we need proof that the consumer side will lead the way that it will stick with them. So…

      Connie: But I think that’s the problem, right? If we’re waiting to have, like, survey data to see if this works, then no platform is going to experiment on it. And this is why, like, new startups and new platforms need to experiment with how to engage with podcasts. I think it’s, like, a given that everyone would prefer to have no ads in their podcasts. And that’s why it’s up to all the platforms to figure out how to create the tools so creators can still make money, and make better money than I think what they’re making now. I actually think creators are vastly underpaid in podcasts. And it’s up to the platforms to figure out how to help them monetize so we can get ads out of the podcast itself.

      Nick: I don’t think we’re disagreeing. I think we’re, sort of, like, coming at it from opposite directions here. Because my number one principle when I’m thinking through these things is that no matter what happens in terms of feature development, no matter what happens in terms of whether certain platforms or tools ends up innovating on these fronts — is whether creators themselves end up controlling their destinies in this situation, whether they control the means of distribution. 

      Like, the entire wave — the entire learnings of what happened with YouTube and YouTube creators — really harms a lot of the people that I speak to when I report week in, week out. Does the nature of the platform being capricious and altering the way that they expect their certain revenue projections over time? And so, I’m personally all for the ability to create better tipping structures, to streamline Patreon and direct revenue, sort of, pathways straight into the listening point. But the fact of the matter is, like, all of these pieces connecting the listener to the creator are all going to be controlled by other people. And I think this is the nature of things that brings the most anxiety to the creator class right now. And, of course, the creator class would change over time with changing expectations of how these things will work.

      Sonal: Connie, I’m hearing you say that there’s huge experimentation that’s already happening in China that we’re not even remotely seeing here?

      Connie: Mm-hmm.

      Sonal: That is also a case, however, where we have platforms, because to the point of tipping, as an example, Nick also mentioned Patreon as a good thing but, you know, clearly, one of the big structural limitations in the U.S. is that people don’t obviously always have their credit cards linked in the way that you have in WeChat, or that we’ve talked a lot about on the podcast.

      Connie: Or like Apple Pay, right? Or, like, in-app payments.

      Sonal: Right.

      Connie: Like, people, oftentimes, will say, like, “Oh, our payment infrastructure is why none of this stuff would work in the U.S.”

      Sonal: But you’re saying that’s not true?

      Connie: And I don’t agree with that.

      Sonal: You’re saying that’s a cop out. Okay, that’s fair. So, then, maybe tipping needs to be done at a more micro level?

      Connie: It’s not even just the money — it’s also helping creators see who their real fans are.

      Sonal: You want the 1,000 true fans.

      Connie: And right now it’s, like, a one-way conversation. Like, why can’t the platforms that allow you to listen to podcasts also allow me to record a quick message back to you? And then also, like, use algorithms to figure out which comments are valuable or not.

      Sonal: Yeah, I think we agree in that sense. Like, platforms should basically do more for their users and experiment. I also agree with Nick, though, on the point that he’s raising. I don’t like the assumption going right to platforms as the default owners of this, and the default aggregators of this. And this, kind of, goes to Ben Thompson, who writes about aggregation theory a lot — which is just a fancy name for network effects in a lot of ways. I mean, he’s much more nuanced. But it is, at the end of the day, the tension between centralization and between bundling and unbundling, and these cycles that constantly go back and forth in waves.

      Competition and centralization

      Connie: Especially with the YouTube platform, like, you look at how the influencers who started YouTube channels 10 years ago, they have massive followings now. They’ve continued…

      Sonal: Yeah. And they’re dependent on YouTube, which is Nick’s point.

      Connie: Yes. But also it makes it really hard for a newcomer to come in and create a YouTube channel and get to that 1 million subscriber count, right? And in the similar way, like, even now I hear about so many friends even starting podcasts.

      Sonal: Oh, yeah. And it’s very competitive. Like there are people who barely get to 10,000 listens per episode and that’s insane. Like…

      Connie: And it’s gonna get more competitive, right?

      Sonal: Yes, very crowded.

      Connie: And so that’s why I think all these new platforms are, kind of, interesting because as they try and pick off creators to have them exclusive to their platform, this dynamic may change. But it’s really interesting. Because, like, for video, it was like winner-take-all.

      Sonal: Which is not true in podcasting. So I’m curious then for your guys’ take, because back to the point of centralization — is to give people a better user experience, and choice, and variety, and ease of use. What do we think about the moves of Spotify and Apple in this space, especially given Spotify’s news a few weeks ago of acquiring Gimlet?

      Nick: So, I think the necessary background here is that for the longest time, Apple has been a primary distributor of podcasting. It used to be somewhere upwards of, like, 80%. We believe it’s now somewhere between, like, 60% to 75% maybe. But with today’s Infinite Dial, so, studies — it suggests that Spotify has grown their particular share. But we’re nowhere seeing like 50/50 parity or something. We’re just not seeing that just yet. 

      And so Spotify — the business case for Spotify going into podcasting, or spoken audio writ large, is pulling their business model away from being completely tethered to the dynamics of the music industry. Which is to say, a music industry that’s been very costly for them to play in. And it’s been very costly for a lot of music platforms to try to come in and take over, essentially, distribution power from the music labels. And so, Spotify looked into the situation and [goes], “We see a category of content here that is significantly cheaper, that is still unwieldy, and it’s still untamed. And we can try to figure out our place in that world and, sort of, push us off the narrative of just being a music company and giving ourselves other avenues of growth.”

      Connie: And that impacts, like, the company’s branding and positioning, right? It’s no longer seen as just a music company, but, like, an audio destination for all kinds of audio?

      Nick: Absolutely.

      Connie: And in that same way, that, like, Spotify was also known for helping you discover stuff you’ll like, I think this is also a reflection they’re realizing, like — podcasting has gotten so large in terms of how many new creators are jumping in.

      Sonal: Can you guys address the exclusive shows angle?

      Connie: I actually see both models working really well. I think if you have a platform where anyone can submit a podcast, that can be great. You can have long-tail creators. But I also think a podcast that says, “Hey, I’m going to curate the top 200, 300 podcasts,” can also work really well, too. Both have great monetization potential if they want to be niche or just long tail.

      Nick: Yeah. And so, I mean, we have a couple of situations that’s pretty interesting right now. So there’s been a paid podcasting attempt for quite some time called Stitcher Premium. It’s a, sort of, exclusive layer on top of a fairly popular third-party podcast app called Stitcher, which is part of Midroll. And earlier this week, we saw the formal announcement of a company called Luminary that’s attempting to be — they literally use the tagline, sort of, “Netflix for podcasts,” which is going to be difficult because the primary challenge there is that they’re trying to build a catalog of things that, you could argue, has free alternatives almost everywhere else. And I have made this argument a couple of times before, and I don’t think it stuck yet but, like — I think we should be looking at Headspace as a really interesting, like, comp here.

      Sonal: What do you mean by that?

      Nick: So, Headspace, essentially, is an on-demand audio app that performs a very specific function that provides a very specific genre of on-demand audio content. It fits into one’s life in a very, very specific way. You know exactly why you’re paying for it. And you can’t find quality alternatives elsewhere off that platform, generally speaking. And so we’re in a situation where there is some lane here to build a paid podcasting platform. The question is, like, will there be a really, really big one, or will it be a series of smaller ones that ends up being bundled over the long run? And I think we are at the very beginning of being able to answer that question.

      Sonal: Yeah, I agree. I would also say, for people in the know in terms of the history of podcasting, in the recent — past five years, I think I’ve seen versions of Netflix for podcasts. And one of them, I remember —I don’t even know if you remember this, Nick, is 60db?

      Nick: I do. Acquired by Google.

      Sonal: Right, they got acquired by Google, and I don’t know what Google is doing inside.

      Connie: But the problem is, like, it’s still a subscription, right?

      Sonal: Why is that a problem? I would love a subscription service.

      Connie: But I think I would rather pay for a specific podcast.

      Sonal: Oh my God, yes. So my number one complaint — so, everyone at a16z has heard my whole thesis on this a million times. Which is, first of all — podcasting is such a homogenous word. We’ve defined it technically, and in user experience. But when I think of the content side of podcasting, I like to split it into a simple taxonomy of three types of shows. There are personality based — what I call cult-of-personality based shows. You know, like “The Ezra Klein Show,” “The Tim Ferriss Show.”

      Connie: Sure.

      Sonal: And my God, by the way, most of them are named after male names. Let’s just not go off on that one. Then the next category besides cult-of-personality shows is what I call, like, more collectives — or like brands, or voices of groups of people, which is what I would consider the “a16z Podcast.”

      Connie: Yeah.

      Sonal: And then the third show is a much more produced, serialized, like, “Serial” or a narrative type of podcasting show. That’s a very loose, broad taxonomy. But if you think of these three categories, discovery for each of them — it is so frustrating to me. Again, going back to this containerization model — that discovery is limited at a show level. Again, structurally, it’s terrible. I keep bringing up structure because while everyone is so caught up in talking about discovery and monetization, they’re missing the big opportunity here, the bigger thing — which is defining a new unit of analysis of episodes versus shows. And possibly even more granular units within that. I hate that we’re still stuck in the legacy ways of thinking about this, when we can bypass things with software. We don’t have to have the CD stage first to get to the individual song stage.

      Then I also talk to analytics people all the time about how feeds limit what tools outside the big platforms can do — like not being able to tag podcasts by topic. Because I believe we all need the ability to find episodes, not entire shows. [If] I like birds and birdwatching, I should be able to find any episodes on those topics regardless of show. Connie, you like real estate and crafts, you should be able to fucking find those topics and discover every single episode on those.

      Connie: But see, this is where transcription, and tagging, and, like, just a much smarter internet native way of displaying podcasts makes all of that, like, automatic. There is no technical reason why we cannot automatically transcribe all the top podcasts. And again, like, I think subscription for, like, an entire platform doesn’t necessarily make sense for podcasts. Like, maybe it’s a good starting point.

      Sonal: It makes sense if you have a collection of shows you like.

      Connie: It’s a decent starting point. But, hey, maybe you’re a podcaster and you’re only going to create, like, a couple episodes, but it’s really, really good content. Right, like why can’t you let people pay for that? And, again, I think it’s not just about the money that’s getting transferred. The problem right now is, like — there are certain podcasts that I would happily pay for and a bunch that I would not pay for.

      Sonal: Yeah, exactly.

      Connie: And right now these platforms don’t give you that option, to say, “Hey, these are the ones that I ascribe more value to.” Much less even just say, “I liked this one, or comment, or anything.

      Sonal: I mean, right. Well, you’re also alluding at, when you talk about the transcription of shows, though, is like — and this is obviously another key point of discovery — is it goes, again, parallel to the web. There was a curated links phase that preceded the portal phase, that preceded the search phase.

      Nick: Just give it a couple of months because Google is working on that. And they are beginning to beta test all of that in terms of transcriptions, in terms of whether a podcast shows — or audio, writ large — shows up in the search engines.

      Connie: But they’re not even going to have all the podcasts, right? The exclusive podcasts on Luminary, Google is not going to have.

      Nick: Well, then that’s Luminary’s problem at the end of the day, right? Like, I think Google’s situation is, is that they’re going to pull in the RSS feeds, or they’re going to pull in podcasts that exist on the open, sort of, ecosystem. And they’re going to transcribe it and they’re going to index it within the search engine.

      Connie: I guess what I’m saying — like, rather than rely on Google as the search engine to do it, at least very basic transcription and search, all the platforms should be able to do it themselves. And like, imagine all the other stuff you’d like to tack onto it. Like, hey, maybe in addition to the podcast on podcasts today, you have, like, five links that the listener can go in and click on…

      Sonal: Click while you’re playing.

      Connie: Yeah.

      Sonal: I would love the ability to embed a link natively, instead of in the show notes.

      Connie: Or a PDF that you can then charge more money for.

      Sonal: Right.

      Connie: Like, hey, to read more.

      Sonal: Right, right.

      Connie: Or maybe, like, all the, like, parts that you cut out.

      Sonal: Yeah, yeah.

      Connie: Like those special clips, maybe someone pays, like, a dollar to un-tap it, right?

      Nick: I agree, I would love to pay for stuff that I want. But, I mean, look, I’m just a normal person that has, like, normal finances. I don’t think I’m going to spend more than X amount of money per month on entertainment goods.

      Connie: I agree that people aren’t going to spend, like, tons and tons of money on podcasts. But I think the better creators would get more rewarded for their content, which means new creators that don’t have, you know, crazy followings to begin with can still get paid.

      Nick: No, I agree. But the question is like, I’ve heard the line of argument that it’s really hard to become a Patreon supporter, or to find a way to give money to a creator that you really support. And I do wonder — the nature of that assumption. There’s only so much frictionless — so much attacking of the friction that we can introduce to that layer — that we find what the most efficient point of, you know, listeners supporting creators ends up becoming.

      Connie: Okay. But that is assuming that I want to support that specific creator. Maybe I only wanted to for that specific episode.

      Sonal: Yes.

      Connie: Maybe I don’t actually want to give the tip to Sonal, but I want to give it to Connie and Nick, right?

      Sonal: That’s fucked up, but okay.

      Connie: I mean, like, no seriously. Like the way that we are thinking about paying, it’s not necessarily the same person who’s speaking even on every podcast. And the fact that we aren’t able to more directly indicate and tie our money to the products that we truly, truly value — I just think that’s [a] really lost opportunity.

      Nick: Well, so, let me push back on that a little bit, right? So the assumption here is that the show is — this show is made up of you, me — and, let’s say, a producer, and let’s say, you know, a couple of people behind the scenes. But I think the reality is that most of the production structures constitute a lot more people than the listener can ordinarily see. So who a listener is moved to tip doesn’t necessarily translate to who is actually creating the content, because there’s an entire, sort of, conversation over here in terms of like how listeners value the creators, how they, sort of, make assumptions about what they want to support, how they want support, why they want to support. There are a lot of gaps of information there to give all that power to the listeners, I think. There still should be some middle point there, in terms of how support works.

      Connie: I’m not saying it can’t go to a show. But a show is — even then, supporting a show is different than supporting a person.

      Sonal: I’m hearing both of you guys. I also hear that there is a lot more granularity you can do because we have an infinite web. And the fact that we define things as containers of a feed, or a podcast, or a show, or an episode — these are all things we can redefine in this new era. And I agree it’s very early innings. I also agree so wholeheartedly that a thriving content ecosystem has to support its creators. And I know you’re arguing for that, too, because you’re arguing in this framework that people have more comments, [that] they have more ability to interact with their top fans. You’re saying the same thing from a different angle. But from a pure business perspective, in terms of being able to run a business that is based on podcasting, there does need to be a middle layer where creators can get the value they need.

      And for me, the open question, quite honestly, is whether the assumption or thesis that happened with blogging — and this is actually the initial premise of Anchor, as well, which Spotify also acquired — is whether there will be now a new wave of mobile podcast creators who don’t have tools. And again, with tools like Descript, which democratize editing, with tools…

      Connie: Right.

      Sonal: …like, just being able to record a podcast in your phone, without having to have, like, a fancy Zoom recorder or mics. Like, that is an open question to me. And I don’t know if people are really going to listen to that, because we have this discovery problem in the ecosystem. And yet there are a few centralized choke points that are coming up now — particularly iTunes, Spotify, Pandora, etc. By the way, on this notion of growing the podcast ecosystem and the total addressable market size, what do you guys make of radio here? Because that has its own set of structural, and policy, and regulatory considerations. I’m curious for your guys’ take on that aspect of it.

      Radio and fragmentation

      Connie: Well, I think the market size for podcasts is, you know, multiples larger than what it is today. And I do think it’s tapping into radio, but it’s also tapping into other things that do really well in the audio format. So, like, audio books that are self-published for example — things that are related to the knowledge-sharing market for adult learning…

      Sonal: Oh, interesting.

      Connie: …I think can really, really work well for audio formats. There’s a lot of stuff where I don’t need to watch someone talking on YouTube with, like, a whiteboard, because, usually, they don’t even really need the whiteboard, honestly.

      Sonal: Yeah. Although, there is a funny argument to be made which is that people also listen to audio [on] YouTube. And, in fact, Chris Anderson was telling me his son watches entire movies on YouTube in audio mode only, which I think is fricking fascinating.

      Nick: I also just listen to movies on YouTube all the time.

      Connie: I mean, yes, YouTube also works for audio. But, I mean just imagine topics around business, topics around finance, topics around parenting — even, like, meditation and how to, like, improve your life — all of that stuff works really well in the audio format and doesn’t necessarily always require video. So, anyways, those kinds of podcasts, at least today, are not the mainstream podcast, right? Because today mainstream podcasts are, again, around shows versus individual pieces. Instead of being, like, you know, a TV show, why can’t you be, like, a movie? And it’s, like, this one-time thing that goes really deep, which is really valuable content? And I think if you take that kind of definition for a podcast, it is so massive.

      Nick: So, let’s begin [with] the whole notion of terrestrial radio, right? Like, it is an industry completely, utterly defined by the nature of the distribution points. It is antennas going out. It hits you in the car, it hits you in the radio. And it commands billions and billions of dollars. My interpretation of that industry and its sort of strange persistence has a lot to do with advertiser relationships. It is still the medium that has the most easy reach for — and that hits the most Americans, and has the most, like, history behind it. And so if you’re an advertiser, you feel significantly more comfortable, because that is your default industry to buy into. And I feel like that feeling of safety and confidence is something that should not be understated. And it’s something that all digital media, sort of, sectors — including podcasting and beyond it — should, sort of, be cognizant of, like, that’s one of the primary things driving that situation.

      Connie: And I think another reason why ads work so well on radio — and it works well on podcasts too sometimes — is it comes in the voice of the creator, versus the voice of the brand or like some other random voice, right?

      Nick: One hundred percent, yeah. The, sort of, buzzword that podcast industry executives use all the time is intimacy, right? And that’s why we, sort of, hear the host-read ad being the pinnacle of the podcasts’ advertising experience. And it’s also its most valuable, like, ad slot, ad type. And so, you know, that’s why like a lot of the genres that you pointed out when you sought to build the taxonomy of a podcast — it’s very personality driven, it’s very people driven. That’s why there’s a little bit of trickiness when we talk about something like fiction podcasts, or non-narrative podcasts, and how you monetize that, how you build that relationship.

      Sonal: Yep, I agree. It’s, very much, native to the content of the storytelling and the medium in that context.

      Nick: Absolutely. And at some point, we will see innovations in business models, innovation in distribution, in the structure. In the sort of, like, you know, container of it that will alter the advertising assumptions here, or the monetization assumptions here. But I still want to go back to — to tie it back to the very first thing we talked about. The definition of it, what we think about it, how we think about it — our assumptions of it being personality driven, or show driven, or episode driven — it needs to fragment at some point. It, kind of, needs to break up because it needs to be a universe that can hold a bunch of different kinds of experiences. In the same way that when we think about television, we’re not just talking about “Breaking Bad,” we’re talking about “Wheel of Fortune.” We’re talking about, like, so many different kinds of styles…

      Sonal: We’re talking about, like, “American Idol,” which is such an important movement around the world when you think of the future of content and TikTok and challenge-based things.

      Connie: That’s why you need polling, right?

      Sonal: Right. But the point is that there is a whole — that was a huge — reality TV, like…

      Connie: Or things around holidays…

      Sonal: Right.

      Connie: …or like the Super Bowl…

      Sonal: Right, special events.

      Connie: …like once-a-year type events.

      Sonal: Right.

      Connie: Like, this is, again, like, we have to break away from show culture.

      Sonal: Exactly. I agree. And to your point just on the terminology thing, Nick, I would say — the word “fragmented,” we use that in the context of industry fragmentation. To me, it’s more, how to make a homogenous term more heterogeneous and have more diversity embodied within it.

      Nick: Yeah. And so I think the question here is sort of like, do we think about the spread as — on the one hand you have prestige TV, and on the other hand you have reality TV? Or do we think about the spread more like — on the one hand you have Netflix, on the other hand, you have Twitch? Like, is that the way we’re going to think about the ecosystem writ large, or are we going to be a bit more specific when we use the term, when we do our coverage? I think that’s also — you know, what we talk about is just as important about how we talk about it. And so…

      Advice for starting a podcast

      Sonal: Do you want to say one more thing and then…

      Connie: No, I want to ask you questions, because there’s so many of my friends today who want to create podcasts. And you created the “a16z Podcast” from scratch to what it is today.

      Sonal: Well, to full credit, it was actually created before I joined. And I took over three months in the production and then hosting it a year later.

      Connie: Okay. But I know, like, the user base massively — the listenership massively grew under your care, so I think you should talk about, you know, what are your tips for someone who just wants to get started on podcasts?

      Sonal: Oh my God, that could be its own episode and I’d love to do that someday. So I guess maybe in the spirit of creation, which is the theme of this episode, I’ll just say some very quick high-level takeaways. Which is, one — and I do this when I give a lot of talks and talk to founders about how to start their own things for their companies…

      Connie: Yes?

      Sonal: …I think the fundamental thing people need to ask is where they are in the taxonomy of shows that I outlined, because that is, sort of, a flow chart for what your next step is for either how to hire, build, or just what tools to use. If you’re a cult-of-personality show, the things you can do are very different than if you’re doing a brand show, than if you’re doing a serialized narrative show.

      So the first thing I always ask people is, what is your goal and what kind of show you want? Because it’s a very crowded environment. So then the next thing is — attention is scarce. With podcasting, maybe less so, because you have a bit of a captive audience in a phone, or a commute, or a workout — or, you know, a situation where they are on a hike or a walk, where they’re only going to listen. But even then, you are competing with other shows, so the number one thing is how you differentiate your show. And one of the number one ways to get a lot of listeners is to have a lot of episodes — a variety of episodes. And so the other way to do it, then, is to enforce seasonality, where you drop a season of episodes — and then just, like, drop them. Like, you know, record 10 and drop them.

      Connie: So, basically, if you want to do it, it’s like a long-term commitment?

      Sonal: I don’t think it has to be, because — as you’ve also talked about — there’s a lot more tools emerging and startups emerging that will allow, like, experimentation and sharing.

      Connie: But for now, it has to be a long-term commitment?

      Sonal: I think Ben Thompson said this — “headcount is the biggest predictor of how much people invest in something.”

      Connie: Yeah.

      Sonal: And I think if a company has people dedicated to podcasting, then you know they’re serious about podcasting. I would say it’s as simple as that. So you do have to invest in it to make it happen.

      Connie: Yep.

      Sonal: But on the simple mechanics, one of the most beautiful things is the thing that I complained about — which is the very thing that also is the best thing about podcasting — is the feed ecosystem makes it so easy to simply record an episode, distribute wherever you want. And then it’s about using the feed ecosystem to then freely put your feed out all into the world because it’s as simple — all iTunes is doing is taking a bunch of feeds. All we had to do when we got on Spotify was, like, feed them our feed.

      Connie: Yeah.

      Sonal: And people can self-select the feeds into different apps, so you can use that to your advantage. And there’s a ton more about the content side. But the one thing I do want to say is that the editing process is now becoming democratized, because there’s a huge gap. I would often put it as the analogy between design and manufacturing, where there is a design phase and a manufacturing phase. And you need to close and tighten that feedback loop to get the best content out. And what’s happening with tools like Descript — you tighten this feedback loop between design and manufacturing, where you no longer have to separate creators and writers from the technical skills of actually editing a podcast.

      Connie: Yeah.

      Sonal: So that’s really important, because there’s a whole bunch of tools now, though, on the analytics side that will — and there are new — a bunch of distribution tools that are now connecting all these pieces and supporting creators. So it’s a very quick answer. There’s so much more I can say on this.

      Connie: I think we need to do another podcast on how to create podcasts.

      Sonal: Well, that would be fun. Thank you for joining the “a16z Podcast.”

      Nick: Thank you so much for having me. I really enjoyed this talk.

      Connie: Thank you.

      • Nick Quah

      • Connie Chan is a general partner at a16z where she invests in consumer tech. She's well-known for her deep knowledge of the Chinese consumer tech landscape and spotting those trends moving from east to west.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      For the Billions of Creatives Out There

      Brian Koppelman, Marc Andreessen, and Sonal Chokshi

      The writer-showrunner is a relatively new phenomenon in TV, as opposed to film, which is still a director-driven enterprise. But what does it mean, as both a creative and a leader, to “showrun” something, whether a TV show… or a startup? Turns out, there are a lot of parallels with the rise of the showrunner and the rise of founder-CEOs, all working (or partnering) within legacy systems. But in the day to day details, really “owning” and showrunning something — while also having others participate in it and help bring it to life — involves doing the work, both inside and out.

      This special, almost-crossover episode of the a16z Podcast features Billions co-showrunner Brian Koppelman — who also co-wrote movies such as Rounders and Ocean’s 13 with his longtime creative partner David Levien — in conversation with Marc Andreessen (and Sonal Chokshi). The discussion covers everything from managing up — when it comes to executives or investors sharing their “notes” aka “feedback” on your work — to managing down, with one’s team; to managing one’s partners (or co-founders)… and especially managing yourself. How to tame those irrational emotions, that ego?

      Ultimately, though, it’s all about unlocking creativity, whether in writing, coding, or other art forms. Because something surprising happened: Instead of TV going the way of music à la Napster with the advent of the internet, we’re seeing the exact opposite — a new era of “visual literature”, a “Golden Age” of television and art. Are artists apprenticing from other artists virtually, learning and figuring out the craft (with some help from the internet, mobile, TV)? And if we really are seeing “the creative explosion of all time”, what does it take to explode our own creativity in our work, to better run the  shows of our lives? All this and more in this episode of the a16z Podcast… as well as some Billions behind-the-scenes (and light spoilers, alerted within!) towards the end.

      Related Stories

      Hallucination vs. Vision, and Selling Your Art in the Real World: Brian Koppelman Interviews Marc Andreessen [written Q&A]

      a16z Podcast: The Internet of Taste, Streaming Content to Culture with Ted Sarandos and Marc Andreessen

      a16z Podcast: The Business of Creativity — Pixar CFO, IPO, and Beyond! with Lawrence Levy and Sonal Chokshi

      a16z Podcast: Belief — An Interview with Oprah Winfrey with Ben Horowitz

      a16z Podcast: Principles and Algorithms for Work and Life with Ray Dalio, Alex Rampell, and Sonal Chokshi

      Show Notes

      • Brian Koppelman’s background and first film project [1:26]
      • Balancing the input of others [10:26] and the writing process [14:00]
      • Getting a movie made [15:20]
      • Managing the producers of a project and advice for talking to powerful people [19:49]
      • Shift toward writers being showrunners [31:08], working with a writing team [35:56], and breaking into the business [40:33]
      • The current golden age of television [43:58]
      • Koppelman’s decades-long creative partnership [47:20] and how he deals with stress through meditation [52:54]
      • Discussion of “Billions” [58:46]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today we have a unique sort of crossover episode with writer, director, producer, Brian Koppelman — who, with his partner, David Levien, also wrote some of the most popular and still discussed movies like “Ocean’s Thirteen” and “Rounders” — which we’ll also touch on in this episode. But currently, Brian is a co-showrunner with David on “Billions,” which airs on Showtime, and the newest season actually drops this weekend. 

      The reason I’m calling this a sort of a crossover episode is that Brian also interviewed Marc Andreessen for his podcast “The Moment,” which you can listen to on iTunes and elsewhere — if you wanna hear more of their thoughts on the difference between hallucination and vision, putting your art or products and yourself out into the world, and more. We also put the written Q&A version of that conversation up, if you wanna read it on a16z.com. But they’re two separate conversations, so you don’t have to have listened to either to follow both.

      Today’s discussion begins with Marc interviewing Brian, and I jump in in between here and there as well. Starting with the business of creativity and the creativity of business. Then going into how to speak to power, speak to one’s team, speak to co-partners — as well as managing the emotions and ego around all that. And finally, ending on some specific moments about “Billions” the show, in the last 10 minutes, where I’ll signal a light spoiler alert warning beforehand.

      We’re here to talk about the business and making of film and TV, and startups, and tech, and the parallels and whatnot. Take it from the top, Marc.

      Getting started as a writer

      Marc: Fantastic. So, Brian, thank you for doing this. So, I’ve always been fascinated — I’m deeply fascinated by the process of creative expression and success, you know, for sure in technology, and we think of what we do up here as fundamentally trying to find the most creative entrepreneurs and trying to help them build, you know, enormous — both creative and professional and business success around what they do. And it struck me for a long time that there are a lot of similarities between how the Valley works and tech works, and how entertainment works — film, television, other forms of entertainment — works. 

      There’s big similarities. There’s also some big differences which hopefully we’ll talk about. You know, <inaudible> has obviously been super successful across both film and television for a long time, and even before that in music, but I wanna focus on film and television. Let’s start with this — what was the first project that you, and I think it was you and your partner, David — the first project that you and David were responsible for creating, selling, and making?

      Brian: It was “Rounders,” for which we wrote the screenplay. And today, there are people online arguing about that movie, which is incredibly satisfying, because, as you know, when you make these bets, it takes a long time to know if you were right very often. And “Rounders” was rejected — it was incredibly difficult, the movie wasn’t a big box office hit. But 21 years later, people are in ferocious online arguments about the most microscopic moments in the film, which back then, of course, I would’ve said two things. I would’ve said, we were trying to make a movie —.write a movie that would have the effect on people that movies like “Diner” had on us, which is that we would watch them over and over again and quote them. And so the fact that that happened is really rewarding, and it was kind of in our minds. But when we set out to do that, we knew that there was only a needle in a haystack chance of success.

      The doing it — we knew right from the beginning, and I think this is something that has been really important to our ability to continue to do this work — David and me, for this long — is, from the beginning, it was only about us getting in a room, or going separately in our individual rooms before we would come back together. And doing the work itself, trusting that if we found a way to do the work itself well enough, some rewards would come. Some have been really delayed rewards, and some have been much quicker. We never seem to know which it’s gonna be.

      Marc: So, let’s start with, for people who haven’t — for our listeners who haven’t seen “Rounders” maybe a thumbnail description of “Rounders”.

      Brian: “Rounders” is a movie set in the poker underground of New York, and Matt Damon and Edward Norton and John Malkovich are the stars of the movie, John Turturro. And it’s about a character who’s faced with a life decision, which is, is he gonna pursue his passion — this thing that he believes he’s great at, even though he’s had setbacks, and in fact these setbacks have threatened his stable life. And so, he’s at a point where he has to choose— the stable traditional road, or the road that his heart is telling him to pursue. And that’s the central question. I mean, the movie has a lot of, sort of, heightened dramatic — you know, you wanna choose a heightened dramatic construct in which to hide the theme, because the last thing you wanna do — if you wanted to talk about the themes, you know, be <inaudible> and just write essays. If you’re gonna tell it in a fictional construct, make that construct compelling, so that only later people are wondering and feel what the themes are.

      Sonal: Show versus tell, kind of thing.

      Marc: So when you say that you do the work, like, what was the “do the work” part of “Rounders” for you and David?

      Brian: First it was about researching. So I walked into a poker club one night, heard the way that people spoke, saw what it looked like, and immediately recognized, “Nobody has made a movie about this. I can’t believe this exists, this should be a movie.” I called Dave. He said, “That’s great. Who are the people in the world that we’re gonna write about? Who are the characters? Who are we gonna care about?” So we started going to this poker club, most every night, taking notes surreptitiously. And then, at a certain point, we felt we had enough of those notes. We started really figuring out what the character’s question would be, who the character would be, what the important relationships would be in his life. 

      And then we had to — so then we started outlining it, and then we had to just decide, “Okay. Starting tomorrow, we’re gonna meet every morning.” One mistake I see people make when they decide they have to do some kind of artistic work, is they think it means they have to grab that identity so hard that it has to shut out the rest of their identity. But what I found was, you don’t have to do that. I didn’t want to put all the pressure on myself of quitting my job and saying, “I need a beret and an easel and I’m an artist, so that’s all I can have.”

      Marc: So, what was your job at the time?

      Brian: I was working as an executive in the music business. David was bartending. And so what we do is, when he would come off bartending he would sleep a couple of hours and I would get up extra early and when we would meet in a storage locker underneath my apartment that had a slop sink in it, because it was like an institutional little room. Had barely room for both of us to sit. I sat on the floor a lot of the time. And we met everyday for two hours in the morning to write the script.

      Merc: And this was purely on spec?

      Brian: Completely on spec. In fact, this is — I think a piece of this puzzle that I never told before, which is that when we had the idea, David met a young producer, and told them the idea and the producer offered us $5,000 and said, “For five grand, I’ll be your partner. I’ll give you five grand, but then we’re gonna share, and if we sell it we’re gonna share in the writer’s fee, and I’m gonna be your partner on the thing.” And we were tempted because it represented, “Hey, wait, someone is paying me to write. We’re professionals.” But we asked some advice — a woman named Rachael Horovitz, who was at Fine Line. She happens to be the sister of Adam Horovitz of the Beastie Boys.

      Sonal: That’s awesome.

      Brian: Rachel was a great executive, and I knew somebody who knew her, and we went and met with her and said, “What should we do? Someone is willing to pay us $5,000.” And she said, “I don’t need to hear the idea, but if someone is willing to pay you guys who have no credits $5,000 now, write the thing and you’ll have a much better chance of success.” And we’ve taken that lesson to heart, still to this day, to write unencumbered. We like to go in a room and let our idea come to fruition fully, let ourselves — let us work out all of the complicated parts of it without outside interference.

      Marc: So let me ask, because a lot of professional — one of the adages, I think, of professional writers is, never write for free. If you write for free, you’re a sucker. You’re being…

      Brian: Well, that was like Samuel Johnson said that, right?

      Marc: Yeah. You’re a sucker or you’re being taken advantage of, right? Never write — you know, a doctor wouldn’t do surgery for free, a pilot wouldn’t fly a plane for free, writers shouldn’t write for free. And I know you’re not writing for free, per se, but, like, there’s an element of this of, like — like, it feels like a lot of your peers need the deal before they’ll write, is that right?

      Brian: Well, but it depends where you put EV, right? You know, right…

      Sonal: The expected value.

      Brian: Right. Where do you put the EV? By the way, look, the way I view the need for personal expression. I actually completely disagree with that quote. I understand what the quote is, it’s talking about don’t be taken advantage of — and it’s also kind of making fun of the artistic impulse. It’s saying, “Are you a professional or not?” But I would assert you can be a professional — you can act like a professional before you’re paid as a professional. It depends how you’re gonna approach it, and it depends on what your expectations are. But our expected value, even though it might have been foolhardy to think so, was that there would be something on the other side of it. And I’ll say this, the expected value of not doing the work is zero. Like, there’s no question about the EV of just thinking I’d like to write and not writing.

      Marc: Well, if you had shown up, and if you guys had just gone and tried to pitch, tried to get an agent — at that stage of your careers, would you have been able to do the project?

      Brian: No, probably not, other than someone would’ve paid us five grand. But then later we did make the mistake of pitching at various times, and, I mean, occasionally a pitch has become a movie for us. But for whatever reason, we’ve found that our strongest work is done in private, and then we take it out and show the world, and that’s — for us, we find that when you pitch an idea. As you know, when someone comes to pitch you, you’re entering into a dialogue about this endeavor. And inevitably, what we found is, a smart person would say something in the room — because let’s assume for a moment that the people across the desk aren’t idiots. Someone says something smart. You can’t help but have that in your head when you’re then going to do the work. And that might be a smart thing, but it really might not be the right thing, because maybe I’ve only explained this feeling that I have about what this thing could be. Maybe I’ve explained it in a way the best I could at that moment, but left to my own, it would take all sorts of different turns, but I have that phrase that the person uttered to me, and I have to keep returning to that for some reason because I’ve already let them inside this process.

      Sonal: I have a question about this though, because, you know, when we go back to this idea of — you have the confidence to do this in private and then put it out into the world — and even with “The Rounders” there was sort of a long staying power that came about with that. It wasn’t, like, an instant, like, box office hit in one weekend.

      Brian: That’s right.

      Sonal: What’s the timeframe that you sort of, A, gauge the success, and B, how do you sort of balance the sort of impetus from executives and other people in your life who care, who are producing and paying for these products, with sort of keeping the creative process intact without over-rotating on data?

      Working collaboratively

      Brian: So, let me back up to answer that question. I have to tell you where I was before we wrote the first thing, and where I was was in a pretty decent state of misery. Because, although I had a job that was well-paying, and on the surface seemed creative — and although I was lucky enough — even having Amy and then our first child was not a salve for the way I was feeling. Which was, like, I wasn’t doing this thing that I knew I had to pursue. I wasn’t doing the work, I was blocked. And I have this notion that when you’re a blocked person, when you allow this creative impulse to be kept down, it dies. And like any other kind of death there’s toxicity that’s attached to that.

      And the toxicity I knew would leach out, and would actually, you know, leach onto the people that I loved because I would become a bitter person. And I want to be the kind of person who would come home and tell my kids that they should chase their dreams with rigor. You know, people often just think of it as a relic of the ’60s and it’s like, “Hey, pursue your dreams. Do your thing.” But it’s like, “Well, wait. If you have a dream, work with incredible rigor and discipline to pursue it.” And so, I finally got to the place where I knew — and it wasn’t about, “Can I have a movie in the movie theaters?” What it was about was, “Can I find a way to have the courage to do the work that I’m worried I’ll fail at, the work that I think is gonna be meaningful?” And so, I decided to follow my curiosity and my obsessions.

      And it’s not merely following your passion. What it is, is figuring out — if I’m obsessed, I’m incredibly curious — if I can get to the root of that and I can somehow create something out of it that is worthy. First of all, in the doing, I will change and become better. So, to answer your question about success. The moment that I was in there for two hours a day, I was charged the rest of the day. So the job that had seemed mundane and bitter, and sort of annoying to me, was much easier to get through, because I had spent two hours already firing on all cylinders. And so, that in the beginning — and of course, along a career, you can hold onto those things and you can let them go. Because we’re all human, which means that we’re all prey to — we can all fall prey to being judged by a standard that isn’t our own, and we have to find a way to remind ourselves that our own standard is the standard that matters.

      So, of course, I’m not gonna say that the whole time I’ve been doing this I only cared about what I felt like when I was doing the work. I will say that each time I have reframed and refocused, to remember that what matters is what I feel like when I’m doing the work, it immediately makes me feel better, and that I immediately don’t care about the rest of that stuff. Easier to say — you might think [it’s] easier to say, because we’ve had this success, but I know I can point to a movie like “Solitary Man” which was a commercial failure, but I mean — it made its money back, but it was not a big commercial success. But I know it’s the best movie we ever made. It got these incredible reviews, so — I wasn’t crazy. 

      That’s how I know — you know, this question that I’m really interested in is delusion vs genius, or delusion vs capability — but I wouldn’t change anything of the four-year struggle to write that movie. And then we directed the movie because, as an artist, if you get to express the thing you wanna express and then you get to make it, you’ve kind of won. The odds against are so great. Even the odds against completing something, right? Even the odds against actually showing up. “I wanna be a writer,” is way different than “I am a writer.” “I wanna be an artist,” is way different than, “I’m an artist.” And we decide when you get to give yourself those designations. But I was so sad, so miserable — and it immediately changed upon doing the work. So I’ve had to force myself to have that be the standard.

      Script-writing process

      Marc: To go back to the state. So, do you ever suffer from writer’s block?

      Brian: No, because I have rituals.

      Sonal: Like morning pages…

      Marc: Could you describe that?

      Brian: Yeah, I meditate every morning, and I do morning pages every morning.

      Marc: What’s that? What’s morning…

      Brian: Morning pages is, like, out of Julia Cameron’s book “The Artist’s Way.” I do three long-hand pages — a real brain dump, where I just let the pen move for three pages no matter what. And it has this incredible effect on me. It’s self-hypnosis. It’s a brain dump, so that you’re putting all the dross — just gets out there on the page. Also, it has the effect of, “I can’t be blocked. I’ve already written three pages.” So you’re in a state of flow. You’re in a state of movement. That is the tool I used to become unblocked when I was 30. And when I was that unhappy, and I said I had to try to write something, I had given Dave “Awaken the Giant Within” and Dave gave me “The Artist’s Way.” And the combination of those things made me realize, I had to figure out what it was that I really wanted to do and be. And then “The Artist’s Way” gave me this tool to try to actualize — and as soon as I started doing those pages, I was like, “Oh, I can do this. I can write. I can actually make good on it.” And I’ve done it for 23 years.

      Marc: Do you keep the pages?

      Brian: My kids have instructions to burn [them] upon my death.

      Sonal: <laughter> Upon your death.

      Marc: So I was gonna say, you know, decades or centuries later these get published as the notebooks.

      Brian: I really can’t. I’ve read Camus’s notebooks and Somerset Maugham’s notebooks, and I’m happy that they exist, but that probably wasn’t their intention.

      Marc: So, what did you get — when you guys sold “Rounders” or got whatever you wanna say — the trigger got pulled. What did you guys have when you walked out of the room to do that, at that point?

      Brian: Well, so we finished the screenplay. It was first rejected. I mean, it’s my favorite story, and I tell it in detail on my blog — which is not a very active blog, briankoppleman.com. But we were rejected by every single agency in Hollywood. One said it was overwritten, another said it was underwritten. I still don’t know what either of those terms mean. And I wrote down everything they all said, and this was an incredible Hollywood lesson because — you know, in the beginning, every rejection feels so personal. Every rejection also feels so final, right? In the beginning. So I wrote down what everyone said, and then we sold the thing, and that Monday — so we sold the thing over a weekend, on a Monday, and by Tuesday, every single agency that had passed called us to try to sign us. And I read them all their comments. I had it on a yellow legal pad, and I just read them. I said, “Well, but you said that thing was overwritten.” I did, I read it to them all.

      Marc: And it wasn’t that the movie had gotten made and they liked it. It wasn’t that the movie was a commercial success, it was simply that you sold.

      Brian: Nothing had intrinsically changed in the work itself, and they all — nobody owned it. Not one of them said, “You know what, I guess I’m…” They all said, “I didn’t read it. It was my reader, it was my assistant. I meant to read it. I read the wrong script that was about poker and I thought it was your script.” It was incredible. But it immediately framed the question for me for the rest of my career about who knows what. So then it’s bought by Miramax, which is something I used to say with pride. And David and I were just the writers, we weren’t the producers on the movie, we weren’t the directors of the movie — but we, and this has to do with continuing to work with rigor.

      There was a moment where they were gonna hire a director who we thought would fire us off the movie, and we thought would do a bad job. We’d met him, we didn’t like him. And so, even though it wasn’t in our billet, we decided we’d better find a director who they would hire, but who would be someone we felt we could work with. And it was really overstepping our position. And I think part of it is — and this gets into — part of it was that each of us were raised in environments where we saw people take these kinds of risks. And my dad was an entrepreneur, and I saw a lot of the time, the way that he would just overstep his position to achieve a result. And so we found out, through some sources, who [the] directors were [at] that the movie company — who they were interested in making movies with. We triangulated that with people we could get to, and found out that our agency represented John Dahl, who was really high on our list.

      And we said to our agents at the time, “Listen, we’re gonna stay in California until you can get us a meeting with John Dahl.” And they were like, “Well, how are we gonna do that?” We said, “We’ll send him the script with a letter that we write, and we’ll just wait around.” And they had all just competed to sign us, right? So this was the very beginning of this relationship with the agent, and in a way, he had to prove himself to us. So we were able to leverage the newness of the situation, even though, often, people in that situation think that they work for the agents. The agents do a really good job of making people feel lucky to have them. But we were aware of the actual leverage in the situation. 

      He got the script to John. John read it. Luckily for us, he really liked it. He came over and met us at our hotel. We all shook hands on it. We knew he was an honorable person. We, then, got to have this incredible moment — which now when I think back on it I kind of can’t even believe it happened — which is, we then called the producers and the studio and we said, “John Dahl is gonna direct “Rounders.'” And they all went, “Well, that makes no sense. He’s supposed to direct this other movie for — how could you do that? You overstepped.” And we all said, “Well, do you want John Dahl to direct the movie?” And they all went, “Yeah.”

      And what was really great about that is — then that allowed us to be on set every day, because when you’re the one who brought the director in and you have this relationship — plus, John has no ego and he knew we understood the world of poker. Also, this incredibly lucky thing was, we were the same age as Matt and Edward, and so there was a relationship that developed right away — which was, we were gonna take these guys and show them the world of underground poker. We were gonna be the experts about this. John Dahl gave us our limits. He was like, “You have to really think carefully about what you say to actors. You can’t contradict me. You have to — we’re gonna work together, but there’s a chain of command.” And with that, he gave us complete freedom. Within that, he was like, “Now help me make the movie.” But none of it would have happened if we would have pitched the movie we would have been powerless. We had ownership because we’d written the whole thing and we’d proven we were experts.

      Talking to powerful people

      Sonal: Can I ask you a quick question on this notion of ownership?

      Brian: Yeah.

      Sonal: David Levien and you guys are both the showrunners for “Billions.” I’m dying to know how — because when a studio buys your show — someone is producing “Billions” — it is your show as showrunner. Like, what if there’s a conflict, and you guys have like a huge falling out — and I’m thinking of the case of, like, the Sherman-Palladinos and “Gilmore Girls,” and they had to exit before the last season, and it totally changed the last season of the show, and then they came back to remake the thing. Is there this thing where you’re owning this thing that other people are now sharing in, and then you have to kind of give up your baby? Like, how does the ownership work?

      Brian: I’ll tell you, it’s so analogous to the way a founder will work with the investors, right? The VC, the board. It’s up to you to manage that relationship. It’s up to you to set the terms. And look, this does get into questions of privilege. Like, as two white men growing up with — David’s grandfather and my father were pretty successful. We learned at a young age how to talk to powerful people. Most people don’t get an education in talking to powerful people.

      Sonal: You’re so right about that.

      Brian: And that —when people ask about advantages, yes, getting college paid for it was a huge advantage — meaning that I knew I could take certain risks that other people couldn’t, because I didn’t have massive debt. But much more important, or certainly equally important, was — from a young age, my dad would, like, put me in situations where I would have to deal with powerful people, and I would have to find a way to get the result I wanted. He would let me be in a recording studio when he was making records, and sometimes ask my opinion in a room full of incredibly scary, powerful people. He would let me be in meetings and he would leave and then I would conduct stuff.

      Sonal: He really set you up for that.

      Brian: And so I understood from a young age how to interact.

      Sonal: How do you talk to power actually? Give us the advice, for our listeners.

      Brian: Well, the main thing is, don’t treat them as — most of the time don’t treat them with the sense of awe and that their station makes them better than you, but also don’t try to condescend to them as though you’re the smartest person in the world.

      Sonal: You’re better than them, right.

      Brian: And, you know, the biggest thing? Make them laugh once in a while.

      Sonal: That’s actually great.

      Brian: I mean, right? Walk into a room, make them laugh, make them feel like you have the answers to their problems, and that you’re comfortable in your own skin. I mean, so much of what I’m talking about is an ingrained sense of comfort in your own skin — is being able to just continue to grow. You must always continue to grow, continue to better yourself — but find a way to sit there in the room relaxed and understand that you’re not sitting there with the all-knowing, all-powerful oracle or Oz. Which is to say, to answer your question — it’s our job to make the show, to make the actors comfortable, to make the crew feel empowered, to make sure the show is written, edited, and shot, right? It’s also our job to make the show on budget, to communicate with Showtime if there’s gonna be, “Hey, guess what? This next week it’s gonna look like we’re over, but here’s how we’re gonna solve that the week after.” Also, make them feel heard when they’re talking about the show.

      Sonal: You’re so right.

      Brian: If they’re giving notes, make them feel heard, make them know that you actually are listening. Then it’s really important that we only take the notes that’ll make the show better, and that we do that in a way that makes them feel good about the process.

      Sonal: That’s fantastic advice. That’s so great, I feel like that can apply to any business.

      Brian: It does. I think that applies across the board.

      Marc: You know how I coach people how to do that?

      Sonal: How do you? Yeah.

      Marc: From “Larry Sanders,” from Artie.

      Sonal: So, tell us. I don’t know Artie…

      Brian: Well, we both love — “Larry Sanders” is like my third favorite show of all time. So, yeah.

      Marc: So for people who haven’t seen it…

      Sonal: I don’t even know what that is.

      Marc: …you must watch it immediately. So, Artie — the producer, played by the legendary…

      Brian: Rip Torn.

      Marc: The legendary Rip Torn played Artie the producer. So, typically, we see this with young people a lot here, which is like, you give somebody — in your world it’s called a note, in our world it’s, like, feedback or, like, you know, “Here’s an idea.” And you give somebody an idea and they immediately get the back up, right? Well they do one of two things — they either take it way too seriously and they, like, try to do everything you tell them — or they get their back up and they get offended, like, “How dare you question my vision?” kind of stuff, and then that sets up a weird dynamic where you feel like you can’t talk to them, right? And both of those are bad, right? One way, you basically hijack their creative vision, usually to bad effect. The other way is you end up with a hostile relationship.

      And so, Artie’s whole approach to dealing with the network executives — and “Larry Sanders” is a show inside a show. Basically it’s a show about a show. His was of dealing with the suits from the network was basically that, you know, they’d say, “Well, I don’t know, you know. I think that, you know, the curtain at the talk show is red, we really think it should be purple.” And Artie would literally say, “That is a really interesting idea. I am really gonna think hard about that one.” And he would write it on his legal pad, and like, “Okay, you know, what else do you have?” And then of course, you know — then in the show, the suit leaves the room, he rips the paper.

      Brian: Yeah, rips up the paper.

      Marc: And the suits are on their way out, and they’re like, “That was the best meeting ever.”

      Sonal: Because it’s a feeling of feeling that you’ve been heard.

      Marc: And so that’s like — what I’m telling people is like, “That’s the baseline.” Like, if you can just do that, you’re better than most. And then to your point, if on top of that you can actually consider and actually absorb some of the feedback…

      Brian: And sometimes listen…

      Marc: …that might be good.

      Brian: …nobody’s perfect. So, there are times I’m working 17 hours a day, and somebody gives me a note I really disagree with, and I might say — you know, as a human, I might once in a while say…

      Marc: That’s the dumbest thing I’ve ever…

      Brian: …”Listen. That’s…”

      Sonal: I tend to say kind of like, “Fuck off. That’s the stupidest idea I’ve ever heard.”

      Brian: Sometimes I say that’s a stupid idea. But here’s the thing, if you have the right kind of relationship with the people with whom you work, you can say that — because they know that’s not your default position and they understand — because you’re in dialogue with them, but not operating from a — no one’s operating from a place of fear, hurt, or misunderstanding. And by the way, if you say that’s the stupidest fucking note I ever heard, call them the next day and say, “Let me tell you what was going on yesterday. Here’s the way I’m gonna think about addressing it.” Or, “Read this and tell me if you still think so.” You constantly have to remember, if you’re in our position, that you’re grateful to be in this situation, but that you’re not an indentured — you’re not so grateful that you’re gonna prostrate yourself and ruin the thing in the process.

      Sonal: Of course.

      Brian: And if you remember that, you’re in okay shape.

      Sonal: The part that I always struggle with, here, and I wonder if a lot of people have struggled with this — is that, I have always had this belief that competency is a thing that will always get you ahead. The result will speak for itself. How do you sort of play back the results to tell the story that you want — because oftentimes, like “The Rounders” example, like — this is the conversation that’s happening around the movie. Because people have ways of defining those things. I think that’s a really big challenge. How do you sort of define it so that you can make sure that the narrative you want told your way — is that part of the point? I mean, in terms of how people perceive your work?

      Brian: Well, when you’re a showrunner of a going concern, you’re gonna get to prove it out or not prove it out because you’re making the show. And I will say, certainly in the relationship we have with Showtime, all their notes are suggestions, and so, Dave and I are getting to prove it out every episode. I will say we did — so, okay, there are a few other things. It’s not a bad thing to learn the mistakes people have made ahead of you. It’s not bad to do research and know, well, what is the third rail in this situation? Right? So if the third rail on the situation is, don’t go more than 3% over budget on a given episode without having conversations. <Then you should know that.> That’s the third rail, then don’t go — then, you know, don’t be a jerk. You’re in an incredibly lucky situation to find a way to do what you have to do. But there are many other non-budgetary examples.

      So, here’s how a pilot works. And when I lay this example out, there will be parallels to your world. So, a pilot gets greenlit. They give you this amount of money to go make the pilot, and you’re in — they’ve already approved the script. You cast the show together. So, that’s another one of these things where you’re trying to find a way to express your opinions, make sure you have the cast you want, while understanding we’re in the real world — you’re not gonna cast a complete unknown to play the lead, unless you have a bunch of other ways to say, “Well, that’s okay because in these three spots we have people who aren’t.” 

      But then, once that stuff’s done, “Guys, go off, make your show.” Right? Because once it starts going, and before it’s edited, there is no feedback they can really give you. You’re making the show. You go in the editing room after you have all this material. You know the show is gonna fit in an hour-long slot, but most people when they cut their pilot, because they don’t actually have the real limitation of an hour, will turn in a 67-minute pilot, because every idea they had, everything they want it to be in there. Now, David and I, because by the time “Billions” had come around, we’d been doing this for a long time. And what happens when you give the 67-minute thing is you’re inviting a bunch of people to tell you how to get the thing to fit the 57 or 58 minutes.

      Sonal: That’s exactly right. The crowdsourcing problem.

      Brian: And suddenly they’re giving you their opinion on it. Also, by you not having to have rigorously— and with discipline — make those decisions, you’ve inevitably left in a bunch of stuff that you shouldn’t have. So, Dave and I decided, and no matter what, we’re turning something in that’s 57 or 58 minutes, maybe 56, if we could do it. We’re gonna take all of those questions off the table before showing it to the people who put up the money. And I’ll tell you, we gave them this cut, and we’re realistic people so we knew all the flaws and the things we would wanna reshoot before it would go on the air. But, you know, they’re gonna make it as to — maybe some of the audience doesn’t know. 

      When you shoot a pilot there’s no guarantee you’re gonna have a series, right? They’ve invested a bunch of money. Showtime’s known for if they make a drama pilot, it’s very likely they’re gonna put it on the air but you don’t know. And so we turn over this pilot and the first thing they said to us when they called us was, “You guys have already done all the stuff that normally takes a month for us to work through with showrunners — which is, you’ve gotten the thing into show shape.” And that’s because we looked ahead at best practice, true best practices. And by the way, it’s hard, right? Actually, when you’re in the situation, you understand why everybody turns it in at 67 minutes. Because you have to — it’s much easier to not have to make those decisions, right?

      Sonal: Totally.

      Brian: It’s much easier to hand those decisions…

      Sonal: It takes a lot of confidence actually, quite frankly.

      Brian: It’s easier to offload those decisions to someone else, the people who are paying for it. Instead, we said, you know, “We’re gonna make these choices and we’re gonna show them that this is the vision we have for the show.” And our structure, I think I’ve put the pilot script online — I think I’ve put it online at the blog. And if you go look at it — I put “Rounders” up there too, which people have really been reading a lot lately. But if you look at it, structurally it’s quite different than the pilot that got on the air. A different scene starts it, because when we got in the editing room we decided, “Well, now we have the opportunity to make the show be the best version of itself.” We were able to gain objectivity, even though it was all of our hearts in there.

      Sonal: It’s only in the edit that you get that arc, totally.

      Marc: And then the one message you’re delivering is like, here’s an incredible product. The meta message which I think you’re delivering is, you guys are professionals.

      Brian: And they said that to us. They explicitly said, “We know you’re showrunners who can make the show.”

      Sonal: You’re the pros.

      Brian: That was what gave — so this goes to your question of the relationship. How do you establish a relationship with them that makes them, “You’re a professional we can trust.” And by the way, as you know, all you want is a founder, CEO, who can not make it your job to run the company and just take the best of your ideas — and you want them to discard the worst of your ideas.

      Marc: Go knock it out of the park. Go do your thing.

      Brian: By the way, those are hard-won lessons over a career, you know what I mean? We were 20 years in by the time…

      Sonal: No, right. You learned that.

      Brian: I think we sold “Rounders” in 1997 and we made the pilot of this in 2015. So, that’s a long period of time over which we figured this stuff out.

      The origin of showrunners

      Marc: So, for people who are unaware, there’s a very interesting kind of split in how movies are made and how TV shows are made, at least these days. Which is, movies are made — generally, the writer writes the script, turns it over, and then other people run with it — and other people being presumably the producers and then particularly the director. The director ends up actually running the project in a lot of ways, right? Maybe with a line producer or something. For TV shows, especially, it seems, like in the last couple decades, you have this concept of a showrunner — and the writers are often, or usually, at this point, the showrunners. And I’m picturing, I don’t know, Louis B. Mayer, or, you know, Jack Warner or somebody, you know, being told that the writers should run the project, and probably screaming and being very upset. Like, that would be impossible. And so, two-part question. What was the left turn in the industry that caused the writers to get in a position where they could be the showrunners? And then, what did you guys do as writers to make sure that you specifically were able to do that?

      Brian: So there’s this great book called “Difficult Men” by Brett Martin that’s about five showrunners — David Simon, David Chase, Vince Gilligan, Sean Ryan, and one other I’m not remembering right now.

      Marc: And this is “Breaking Bad,” “The Shield,” “The Sopranos.”

      Brian: That’s right.

      Sonal: “The Wire.”

      Brian: “The Sopranos” and “The Wire.” But he goes into the history of it, and “Hill Street Blues” is when this first — because they were making this kind of serialized show, and Steven Bochco started having meetings with the directors. When the director would come in, he would start having meetings saying, “Let me set the tone.” He was executive — nobody named him showrunner, but he decided that he was going to — had to, because of the nature of that show, exert upon the situation a kind of tone — a control of the voice and tone of the series.

      Marc: Because most shows, the successful shows had been more like, “Law and Order” was like, the apotheosis of the other way around — which is, each episode is independent.

      Brian: Yes.

      Marc: Right. More or less. Before “Hill Street Blues.”

      Brian: “Hill Street Blues” was one of the first shows that sort of combined these elements for a cop show, I think, for sure. But the answer to your question is, about Dave and me, and about anyone who wants to be a showrunner — which I’m happy that showrunner’s officially in the dictionary now. Like, two years ago it became — in the dictionary because it’s a real…

      Sonal: I’m so glad. I love that word.

      Brian: Yean, because it’s a real — yeah, it’s a real job title now. Like, what do you do for a living? Showrunner. It’s learning to be a producer, and we have 150 people who will work with us, but we’re in charge of. And it is quite different, but, you know, as you know — David and I directed movies and we produced movies, so, for us, it was quite a natural thing, because we’d already — you know, “Rounders” was as good an experience as you could have as a writer, and there were still areas in which we didn’t have enough control over the the voice. And what we also knew was, we’re probably never gonna get that exact situation again, so we’d better learn how to do these other parts of it. We better learn how to gain control of the, you know, mechanisms of production.

      Marc: The means of production.

      Brian: The means of production, that’s exactly right. And so, we realized that we ought to do that. But, again, that goes back to this question — often a writer takes solace, while they’re whining about not having control, they take solace in not having control, because, if you don’t have control, you don’t take the blame.

      Marc: Somebody else just will.

      Brian: So, if you’re comfortable, if you can find a way to be comfortable with failure — which as a writer you have to, or comfortable in your mistakes — then you can be comfortable in wanting to be the final voice on what the product is gonna be. And we very early on decided — and I’ll say this, when we work with Steven Soderbergh, we are so glad to have his voice. If he’s directing the movie, man — what a thrill to work with a genius, right? And what a thrill to have Soderbergh make us better. To this day, like…

      Marc: This was “Ocean’s Thirteen?”

      Brian: Yeah, but also “The Girlfriend Experience” and then he produced “Solitary Man.” I mean, if Steven called tomorrow and said that he wanted us to just be screenwriters on a movie he was directing, we would jump at it because he’s gonna make our stuff better. But if you’re comfortable taking the blame, if you’re comfortable in a position of control, it makes you incredibly comfortable to then cede that control — or to share with somebody else. And so you can pick your spots then and decide. And also, because we’re able to make our own stuff, being in a situation where we are not the final voice doesn’t make us chafe against it. But I have plenty of that over here, so I don’t have to chafe against it over here. I’m happy to play this role in this situation.

      Sonal: Fantastic. I love this.

      Brian: That’s why we’re good producing other people’s movies, when it’s someone else’s vision — we’re great at just helping them achieve their vision. Like, Neil Burger, who’s an incredibly successful director. Directed the pilot of “Billions.” We produced his first three movies. And I was like, “Hey, Neil. We’re here to advise, counsel, help. It’s your movie, go run with it.” We’re comfortable in any of those different modes creatively, but I think the reason for that is that we got comfortable early on with just doing the work and failing.

      Working with other writers

      Sonal: That’s right. We’ve been talking a lot about kind of managing up — hierarchically, so to speak. Now, turning it the other direction, like, managing down in the writers room. You’ve got like a lot of writers working with you, so how do you now navigate debates with all those writers in the writers room? Like, essentially, you’re the showrunners — how do you make it collaborative yet not a democracy at the same time?

      Brian: Well it isn’t a democracy. So, different showrunners approach the question of the writers room differently, and some who’ve come up through a writer’s room rely on it in a very deep way.

      Marc: You have to describe a writer’s room.

      Brian: A writer’s room is, you get, let’s say, six people in the room, plus a showrunner. There’s a white board on the — and you start at the beginning of the season and it’s like, “Where are we and how do we fill that in?” And then each — but then it’s really hard to describe a writers room, Marc, because writers rooms become extensions of the way the showrunners see the world, and the way they see the world of their shows.

      Marc: In theory, it’s a team of people writing the show together in some form.

      Brian: In some form, meaning maybe everybody will write an episode. Almost all shows, the showrunner does the final pass on all the episodes, no matter whose name is going on.

      Sonal: Yeah, like the top edit.

      Brian: On our show, though, David and I end up writing most of the show, and we have a great room of men and women who help us really break the story arc of the season, and that is an invaluable process. Tons of stuff comes out of the room about how the big arc of the season should occur, about the twists and turns, about where characters — and that’s a months-long process of talking. We haven’t yet found — and then, when it comes to writing the scripts, David and I — and then we have a writer named Adam Perlman, who’s now a co-executive producer — he’s our number two person, and Adam writes a good amount of the show, too. But the truth is, it is mostly us writing it. And I’m not saying that’s the way it should be on every show. The voice of our show, the way that our show is — whether you like our show or not, our show is canted in a certain way. It has a very clear voice that somehow the two of us can do. Now, that said, when someone else —if someone on the team starts a script, their name goes on and ours does not.

      Marc: So you got a young hotshot writer and they have an opportunity to write on a show that’s maybe not as, let’s say, critically respected or whatever, but maybe it’s like they know that they’ll actually get to write scripts and…

      Brian: Yes.

      Marc: What’s your pitch to them of why they should come work for you given that it’s a more constrained environment?

      Brian: I’m not sure. Well, Adam was somebody who had a lot of job offers when he came on our show in the second season. He started in the second season. Came into the room as just a producer-level, which is kind of a low level position, in terms of the hierarchy. He wasn’t helping to be a showrunner. But he came in the room. He had incredibly good ideas. He then wrote — his first script that he wrote was very strong — strong enough that when someone’s script came in that was not that strong, and David and I had to work on three other things, we called him in and we said, “Hey, take a shot at rewriting this. Here are the things that matter. We made extensive notes. Adam, go try to rewrite it.” He rewrote that script. We then sat with him and talked about how we were gonna now rewrite it, but he did a really good job. We kept being able to go to him, and by the next season, season three, he was running the room when we weren’t there. We bumped him three positions — we bumped him up to co-executive producer really quickly and said like, “Look. You’re our creative partner now. Like, help us do this.”

      So, if you’re really great — if you’re great in the way that our show requires. Someone may kill it on another show and just not kill it on ours. I mean, the other thing is, they get to be on set, watch how our show is made and be a part of it. I’ll say one thing though to answer — another thing to answer your question, which is, some people have come into the writers room, talented — and I found out they came into the writers room because they like my podcast. But I’ve had to say to them, I’m this incredibly nurturing and encouraging voice on the podcast, and I want you to know, like, I am that for you in your life, and I’ll, like, help you get the next job and I’ll be — but you’re gonna turn in a script and you’re not gonna get the voice on the podcast.

      Sonal: Oh my God. Totally relate to this.

      Brian: You’re gonna get somebody saying to you, “Here’s what doesn’t work.” And so you have to know that this is now you’re entering — we’re in the major leagues here, we have no choice because we’re playing the Red Sox tomorrow, so we have to be ready to get in there and play the Red Sox. That has happened twice.

      Marc: So, one more question about “Rounders” which goes to the kind of current state of the of the industry. So “Rounders” was made in when, what year?

      Brian: ’97.

      Marc: ’97. So, that was the heyday of, kind of, the high-status independent movie, like, medium budget but like super high status.

      Brian: Yeah, 14.5 we made that for, yeah.

      Marc: Okay, yeah. And then as you said, like, you know, it wasn’t a huge commercial hit out of — but then it had this long life, you know, kind of, you know, plays out through now and probably long into the future. If that movie had not gotten made, and if movies like that had not gotten made — and just nobody had made, kind of, the definitive poker movie, and you and David entered the industry today at age 25 or 30, or whatever it is, and decided to make that movie or that project today, what would be different about the process?

      Brian: People constantly ask me how to break into the business, and my answer is — I have no idea, I did it 23 years ago. I can’t help you. I wish I could help tell you how to break in. The conditions on the ground are entirely different, and the last thing I wanna be is some general, back in [the] thing, ignoring what the sergeant says. Like, I have no idea. I do know that — what I know is that — well, I think it would resemble a movie that I love and that launched many careers which is “Margin Call.” But whereas “Rounders” was a 14.5, I think “Margin Call” was made for $1.2 million, and scraped together by a commercial director and they had limited sets. Because “Margin Call” has a lot of similarities to “Rounders.” It’s set in an insular world with a language of its own. It doesn’t spell anything out for you. Like, you have to be willing to roll…

      Marc: We should describe — it’s kind of the definitive movie of the September 2008 financial meltdown. It kind of takes place overnight, effectively in Lehman Brothers, and it’s like a very fictionalized version of Lehman Brothers. And it’s actually a very chilling — the people in finance look at it and say…

      Brian: Goldman. Well, it’s Goldman, right? Because they survived. I think it’s set at Goldman, and it’s about willingness of Goldman — and they never say it’s Goldman — it’s about the willingness of — it’s about a decision that Goldman Sachs made to get rid of their toxic assets. But I think that movie is really analogous to “Rounders” because it is doing a bunch of the stuff that we did. It’s — you have to just, like, catch on to the lingo and you have to understand what the stakes are, but you had to — look, they made that movie for a tenth of what we made “Rounders” for.

      Marc: They had Kevin Spacey, they had Jeremy Irons, Zachary Quinto was in it and he was starting to become famous.

      Brian: Yeah. That’s right. Yeah, that’s right.

      Marc: So they had top-end.

      Brian: They put the top-end people, but it was still they had to make it for, like, a million and a half bucks, a million two maybe. It’s much harder to make those sort of mid-budget, $14 to $25 or $30 million dollar movies, though Netflix does it, right? You can do it at Netflix now, which is probably where it would happen. Or you would try to tell the story in a novelistic way, you know…

      Marc: That was my question. So would you pitch today, young David and young Brian show up. Would you pitch “Rounders” for television or for a film?

      Brian: No. You would pitch the world of the underground card rooms for television.

      Marc: Okay.

      Brian: Because I think a lot of that — that’s where this stuff lives and that would have been I think a fascinating thing to see also. David and I grew up watching movies. We loved television, but our shared language, our lingua franca, was movies. We were quoting movies at each other from when we were little kids. We would watch movies 20 times, you know. We watched “Stripes” together at least 20 times, and “Diner,” and many more movies where they became the way we communicated. And so, it made sense to us to go make movies. Since then, you know — things like “The Sopranos,” “West Wing,” “Larry Sanders,” “Mad Men” showed up and showed us the way. They lit the way, sort of, for us to think about television.

      Sonal: Yep, that’s actually huge. We always talk about this, Marc and I. Television is so much better than movies, it’s unbelievable.

      Marc: Well, I think it’s actually — the best shows, they are novels.

      Brian: I think we all think of it…

      Marc: …or series of novels.

      Brian: You think of them that way.

      Sonal: I call it visual literature.

      Marc: The movie is still more like a play, whereas these shows can actually — these shows are like thousand-page novels.

      Brian: We definitely think of it that way. We’re trying to tell novelistic stories, deepening characters in challenging situations.

      Sonal: I call it visual literature, it’s exactly what it is.

      Brian: I love that term.

      Golden age of television

      Marc: So you came up in the music industry. You know, I was involved in the — <laughter> the internet, and then, you know — I wasn’t involved in Napster, but I knew the other guys really well. And so, we both watched, you know, from various professional perches, kind of — the music industry confront digital distribution and basically just, like, implode, right?

      Brian: Oh, yeah, get run over. They didn’t confront it. Unfortunately they didn’t confront it. They just stood there — they just got run over.

      Marc: Kablooey, right?

      Brian: I mean, like, France. They’re in the deuce man, you know. They were like, “Should we pick up our guns and rifles?” “No, let’s just lay down.” <laughter>

      Marc: That’s it. That comment <inaudible>.

      Brian: No, Marc, you signed off. You laughed. You completely laughed.

      Marc: So, I fully — I’ll just confess, I fully expected the same thing was gonna happen to TV. Like, you know, you use Napster for music, BitTorrent for TV, and it’s just like, it’s just obvious — same thing’s gonna happen to TV. It’s just gonna get run over. And then the most, like, amazing thing in the world happened, which is — the exact opposite thing happened. The opposite happened, which is, like, the creative explosion of all time. And you’ve probably seen — you know, John Landgraf who runs FX is always talking about that…

      Brian: He’s a brilliant man.

      Marc: Brilliant man, great programmer. But, you know, he talks about the content bubble, the TV bubble — and it’s like, I don’t know, every year now it’s like 500 original scripted dramas are getting made.

      Sonal: I think it’s 560 or something insane.

      Marc: Yeah, and he’s been calling this a bubble the whole time, but, like, it keeps expanding and I mean, we all get to, you know — you get to make it, but like we get to watch it, and it’s just like — I, like, routinely see shows now where I’m just like, you know, 20 years ago this would have been the best show in the entire history of television.

      Sonal: Yes.

      Brian: The fact that “The Mindhunter” and “The Crown” came out, like, in the same year on Netflix is amazing to me. Those would have been the best show of an era…

      Marc: Ever.

      Brian: …ever. Like, “The Crown” is as good as you can make something.

      Sonal: I keep trying to make Marc watch it. He hasn’t…

      Brian: I can give you the language by which to watch it. So I’m totally not interested in monarchy. I hate it, and nothing about that is interesting to me. The show is just the most beautifully written and shot and acted show that there is.

      Sonal: I agree with you. He doesn’t believe me.

      Marc: So here’s my question. Let’s assume it’s not a bubble. Let’s assume it is the medium of our time, and let’s assume it kind of keeps expanding so this all makes sense. But the amazing thing is, it seems like the more shows get made, it seems like the average quality level is rising. And you would expect, I think, the opposite. You’d expect the average quality level to fall because you’d expect to run out of talent at some point.

      Brian: I agree.

      Marc: And so, where is all this talent coming from?

      Brian: I have no idea.

      Marc: So, were there just all these geniuses out there who just never had the opportunity to do it, and now they do? Or is there something happening in the industry where people are being trained in a different way or…

      Brian: Or maybe it’s the love of television, so it perpetuates itself, and we might be in a golden age where artists are apprenticing in some way for other artists, and learning and figuring it out. You know, I have the luxury not to think about the 560 shows. Or to appreciate what Landgraf says, and know he’s a brilliant guy, without having to be cowed by that. Or feel any way about it, because I just wanna — I still go back to the same thing, I just wanna get in the room and get the feeling I get when I’m making the thing. I wanna be able to walk on the set and see Damian and Paul and Maggie and Asia, and be able to work with them. And, you know, we’ve just found a way to make decisions still based on our curiosity and our obsession. So, if we’re interested in the U.S. Open in 1991 and Jimmy Connors, we’ll go make a documentary about it, because we’ll really enjoy the process of making it, and we have faith that there will be people who will wanna see it.

      Sonal: I was thinking of my answer to Marc’s question. I’m trying to make him watch this movie “Gully Boy.”

      Brian: I haven’t seen it.

      Sonal: It’s a Bollywood movie that’s produced by Nas, but to me the point is that technology has democratized the access to watching all this visual literature.

      Brian: I don’t understand — Ben is not able to make him watch something produced by Nas? That makes no sense to me.

      Maintaining a partnership

      Marc: Ben and I have the kind of partnership where we’re able to, you know, we’re able to complement. Actually, I wanted to ask you — that was the other question I wanted to ask you. So you have been partners now with David for how long?

      Brian: Over 20 years.

      Marc: It’s an equal partnership?

      Brian: Has always been from the beginning.

      Marc: Okay. Equal partnership.

      Brian: Fully 50-50.

      Sonal: Beautiful.

      Marc: So how do you — if somebody comes to you and says, like, I wanna have a partnership like that. I wanna have a career where I have a partner like that. Like, how do you do that?

      Brian: Well, do you remember when the four of us first met, how funny it seemed? When me, you, Ben, and David — we were sitting there and it was just like, “This is a rare thing to have two sets of people who just…” In the same way it makes sense when someone sees you and Ben and talks to you for five minutes. When someone sees David and me, and they talk to us for five minutes, the whole thing just kind of makes sense. Like, in the ways that we can finish each other’s sentences, but also are different in some significant ways that probably we don’t — like, if someone else heard us talking, we’re maybe very similar, but the two of us understand the ways in which we’re complementary to each other.

      The key is to really regard the other person as incredibly smart, to really always know that their motive is to make the work better. So much of this stuff sounds like platitudes, but like — trying your hardest to get your emotions out of these decisions and being rational. I think the key to having a good partnership is not about looking for the partner — it’s about how can you make yourself be the best version of yourself in a way that complements this other person, who you respect and whose work you admire. And so, that’s all hard work in life, right? It’s the same thing in a marriage and any kind of a partnership. 

      But it’s about all of us — even the most rational, the smartest among us — have emotional reactions sometimes. And the question is, okay, it’s not to not have an emotional reaction but it’s to not let the emotional reaction dictate your response. So if that means you know that you normally — the worst of you instantly reacts with anger, then find a way to say, “Hey, I don’t wanna react with anger. I’m gonna go take a run, and then I’m gonna come back.” And this is stuff you figure out over a long period of time. But the more you know that the success or failure of a partnership is based entirely on how you comport yourself, the better off that you’ll be.

      Marc: It’s not the other guy’s fault. That’s right, it can’t be the other guy’s fault. You have to take the responsibility yourself

      Brian: Don’t you think of it that way…

      Sonal: I actually am curious what Marc’s take on this is.

      Brian: Yeah, what is your take on that?

      Marc: No, so, the way I describe — by the way, this comes up a lot in our business. You know, Ben and I have this kind of partnership — lucky for me — but also, you know, there’s a lot of, like, founder and then CEO. Like, sometimes we have founder CEOs, which is like your showrunner model, but sometimes we have a founder and there’s a CEO who’s brought in, or promoted inside the company, and then they have to be — you know, if you want the magic of the founder and the company to be well-run, they need to have that kind of partnership. 

      And so what I always tell them — I kind of try to put a point on it, and just kind of say — it has to be more important to each of you that — it has to be more important to each of you that the other one — how do I put it? It has to be more important that the other one gets to make the decision, than that you get to prove yourself right. And you have to both have that attitude. Like, if one of you has that attitude, then that person is just gonna run over the other one. If you both have the attitude, where your reflexive view is, “You know what, this is a debate, it’s an argument. It’s 50-50. It’s a toss-up, which a lot of these things are. We’re gonna do it your way.” If both people have that as their default point of view, then you can navigate through these things. And then you get in the positive version of the deadlock, which is…

      Brian: Yes, you’re totally right.

      Marc: Let’s do it your way. Okay, now we have a healthy conversation going, right?

      Brian: You’re totally right. Sometimes there’ll be emails back and forth about a thing in editing, where one of us will have an idea and the other one will say, “My instinct was to go the other way with it, but you know what, let’s do it that way.” And it’s not even — it has to not be a move, I think, you have to actually be like…

      Sonal: You have to really be into it.

      Brian: …well, all right, let’s — there was a thing yesterday where I saw something, and I had a notion about it, and David sent me back — well, there are a few different things that are good. So normally, when we’re doing edits on — when we’re making notes on a cut in order to do edits, our two assistants — we share two assistants, it’s not like one’s his assistant and one’s mine — we have two assistants who help the two of us. Normally they’re on the conversation, so that they can then collate the notes and give them to the editor before we go talk to the editor. But if there’s something that suddenly is gonna — we see really differently, we just immediately take it to a private communication, right? We take the audience out of it. We never talked about this, but we just do it. We take the audience out of it because we’re not performing and we’re also not worried about being judged.

      But so, yesterday was one of those things. We just saw one little tiny moment slightly differently. I wrote this thing, like, I think we should do this — and then Dave wrote me separately and said, “You know, I don’t see the scene that way. Here’s what I think is going on.” And I still saw the scene the way that I saw it, but I just immediately went, “No. Yeah, let’s just do that.” And it makes total sense. Like, let’s go through the next bunch of iterations of the cut with it in like that, in the hope that I’m just gonna come around to seeing it that way. Or let’s show it to some other people this way and let’s see what comes out of it. It would have been very easy, and I see a lot of people fall into the trap of trying to argue.

      Marc: Well, I look for as many — by the way, I think about it as — I look for as many chances as I can to let him make the decision, right? And then, to your point, like, if I really feel — and as a consequence of that I build up so much trust.

      Brian: That’s right. That’s what I’m saying.

      Marc: That if I feel strong about something…

      Brian: Well, that’s a really great point. This is important to attach to that, which is, because all of the time Dave is willing to say to me, “Let’s do that.” When he wrote me and said, like, “Hey, I think this is different than you think it is.” It was just so easy to go, “Well, of course, dude. Let’s do that thing because you’re always looking to let it be the way I want it.” I would say I’m certain none of that is a tactic or a strategy with Dave and me. It just so happens to be the way that the two of us interact.

      Meditation and dealing with stress

      Sonal: A quick question on this though, just from, like, an advice point of view, because you talk about this. How do you manage your own personal psychology around anger and creative impulse and ego, kind of in this process, even beyond the partnership?

      Brian: Well, meditation helps. I mean, I know, as I said before, some of this stuff sounds so reductive, and so much like platitudes — but, you know, I love that Tim Ferris has said, out of the whatever thousand people he’s interviewed who he views as highly successful creatives — like, 92% of them meditate. And I don’t think that’s just buy-in. I don’t think it’s just that everyone’s decided to buy in.

      Marc: So I’m in the 8%.

      Brian: Yeah, I know.

      Marc: I’m like mister anti-meditation.

      Sonal: I’m not into meditation.

      Brian: You’re anti-meditation?

      Marc: Well, I’ve never. I’m not philosophically anti-meditation, I’m personally anti-meditation. I cannot imagine sitting still with my own thoughts for longer than about 30 seconds.

      Brian: I couldn’t either originally.

      Marc: So this is my question. So, talk to me as a practical person who’s interested in performance, and not particularly interested in introspection, like, how would I…

      Brian: Well, I do the simplest kind. I do transcendental meditation, so it’s the easiest one, because it’s just quietly saying a mantra to yourself for 20 minutes.

      Sonal: Yeah, define transcendental meditation.

      Brian: Well, transcendental meditation is you — because I — ADHD person, I can’t sit still, I have to check my — all that stuff. Except I really do this twice a day, 20 minutes. And what I found — and it’s just personal, but what I found was it, like, reduced the physical manifestations of anxiety by a lot. And for me, when I — getting anxiety out of the equation, I just think more clearly and more creatively. And it’s not — I would say, the other thing is people build it up too much, right? It’s not some magic pill. It doesn’t, like, immediately make you…

      Sonal: In a state.

      Brian: You’re not suddenly becalmed, but it just kind of takes, like, a little bit of the tumult out. And a lot of forms of meditation require you to force out the thoughts, as you said, require you to be introspective, or require you to focus on your breathing. Transcendental meditation — all you’re doing is sort of allowing this mantra to be said over and over, and if thoughts come in, that’s fine, you just kind of let the thoughts come in — and then you kind of return to this mantra. And I would say the results for me — so I was hugely skeptical, but I was at a point where I was feeling like I needed something. I had too much agitation. And so in reading — I read David Lynch’s book “Catching the Big Fish” and a couple of other books, and it made me interested enough. And I went and sat down with Bob Roth who runs the Lynch Foundation and I said, “Look. I think you’re probably a cult. I’m an atheist. You know, I know these are like Sanskrit words that have some holiness to them. So, none of that stuff works for me, so talk to me about why I should even be sitting here.”

      And, you know, Bob was like, “Well, why don’t you read this book, and why don’t you read this study, and why don’t you look at these EEGs, and let’s talk about what this tool does in terms of affecting the loops in your brain and your brain waves.” And through that conversation, I was like, “Well, okay. I’ll learn.” And within — I’ll say, like within two months I noticed, and my family noticed, that I was just in a much better place. And, again, it doesn’t mean I’m never a dick. Like, we’re all a dick sometimes. It doesn’t mean I’m never short with anyone, or that I’m never worried. Of course I am, I’m a human being. But it means that I can manage it in a much better way, and if the only thing I got out of it was, I was sitting and meditating — and when you’re not trying to think of ideas, but like — I’ve solved many tricky story problems. I’ve come out of a meditation, and just kind of had the answer show up. Now, that could just be a function of, like, I turned everything off and I consciously wasn’t thinking about it, and so I allowed…

      Sonal: Your mind went to work.

      Brian: That’s great. Perfect, whatever it is. It’s not surprising to me that so many of us who are high achievers, aggressive in going after what we want, willing to take risks — that finding some tool that gives you some enforced break from that — it’s not surprising to me that then when you then come out of that, you’re kind of firing again, right?

      Marc: So recharged, reset.

      Brian: That’s just what makes sense to me about it.

      Marc: So who’s Bob?

      Brian: Bob Roth runs David Lynch Foundation. David Lynch Foundation is, like, at the center of transcendental meditation. Lynch had decided — the real David Lynch.

      Marc: The director, David Lynch?

      Brian: David Lynch is the reason transcendental meditation is popular in America. Lynch credits TM with making him the artist that he is.

      Marc: David Lynch just for — Twin Peaks, Blue Velvet.

      Brian: Oh yeah, all that stuff. He started doing, like, 40 years ago or 50 years ago, and he wanted to start a thing that would give it to kids, and post-traumatic stress people, so he started this foundation and the guy who runs it and who’s like sort of the kind of the head of TM in America is this guy, Bob Roth.

      Sonal: The best part of that story, by the way, though, is that you’re literally arguing — to Marc’s point, about this — because Marc essentially set it up as a tension between performance and introspection, and you’re essentially arguing that introspection leads to better performance, which is what I love about it.

      Brian: Well, no I would argue that it’s not introspection. Like, my journaling is definitely a certain kind of introspection, it serves me. But meditation is, like, the calming of the thoughts, or the stilling of it. Or it’s just a respite, in a way. It’s a respite from the perpetual thinking machine thing. I think the idea is that you have these thoughts, these pattern of thoughts, and there are some thoughts that you know you have. But then there are these, like, patterns of thoughts that you have that are probably a little bit disruptive, but they’re a loop. And when you start to say this mantra, you’re interrupting, right? Suddenly, that’s what the sound is and the other thing just dissipates, and you get calm. I’m not trying to think about my life when I’m meditating, I’m just trying to take a break.

      Talking about “Billions”

      Sonal: Yeah, okay. Let’s spend the last few minutes just talking about “Billions” specifically. Podcast friends, we’re about to go into some light spoiler alerts — particularly from the last and early seasons — so if you haven’t seen them already, you’ve been warned. I have to ask this question, because you know that scene from “As Good As It Gets,” where there’s a female character that goes to Jack Nicholson and…

      Brian: Yeah, I take away honor and — what’s the exact line?

      Sonal: Well, actually I was thinking of another thing…

      Marc: You guys gotta — I have not seen this movie, so.

      Sonal: Oh, you haven’t…

      Marc: You guys have to describe.

      Brian: I thought you were gonna say the one where, “I think of a man and then I take away reason…”

      Sonal: Yes. Well, that was his response to her, because the question that I have is how do you write women so fucking well?

      Brian: Well, that’s his answer.

      Sonal: And that’s right.

      Brian: I disagree — wildly disagree with his answer.

      Sonal: Which is good to hear, but the best characters on “Billions” are, quite honestly, the female and transgender characters of Maggie Siff, who plays Wendy Rhodes, and Asia Kate Dillon, who plays Taylor. I mean, I want to ask you, how do you do this incredible character development for these female characters?

      Brian: You know, the hardest questions to answer are the “how do you do the thing.”

      Sonal: I know.

      Brain: Because that’s the part that’s not — there is no intellectual answer to that question. That’s the part of it that either makes you someone who does this, or doesn’t do it. The most fun part for me is when I’m sitting on my couch, actually writing the scenes, right? I have music blasting, able to put the computer — the laptop actually on my lap — and I’m able to sort of fly. And that’s the part that isn’t intellectual at all. It’s the result of all the intellectual work you’ve ever done. It’s the result of your curiosity, it’s the result of everything you’ve read, of everything that you’ve watched, of everything that you’ve been a part of. And then you want to just allow it to happen. And so, we honor these characters — and Wendy Rhodes, when we, you know — invented that character and then wrote her, we certainly know who that person is very well. But you have to make these fictional characters feel incredibly real to you, and you wanna write them smarter than you are, and that’s the only thing I can say is — we want every character in “Billions” to be smarter than we are.

      Sonal: So, a quick question about Taylor as a character, because “Billions” — the next season is now dropping. You ended the last season with a tension between the head of Axe Capital and his protégé, Taylor, starting their own firm. And I so relate to Taylor’s character like you won’t believe. There’s a sense of, like, unbounded ambition.

      Brian: Are you trying to tell Marc something right now?

      Sonal: No, no, no, not in that sense.

      Marc: This happened before.

      Sonal: There’s an unbounded ambition with Taylor, and Axe initially nurtures it and then essentially squashes it. I’m dying to know, like — Taylor is a really interesting archetype actually. Both that Taylor is transgender, and that you have this essential universal archetype in every organization. Tell me how you think about Taylor as a character.

      Brian: Well, Taylor’s just the most highly competent person, and is a brilliant person — and, like, if this is a long novelistic piece, we’re still sort of at the middle — the beginning of the middle of the story. And so, that kind of person has to be tempted, right? Has to be tested. If you don’t test the morality of those kind of characters, how do you know whether they’re really moral or not? If they don’t get lost for a little while, how do they become found? And so, that’s where we find Taylor in this season. I don’t wanna spoil anything.

      Sonal: Okay, I have another quick one I’m just dying to ask — and we’ll lightening round these and then we’ll wrap up. I wanna ask you about some of the music choices you make, and one specific one. Last season, one of the most compelling, raw music choices you made is in a scene — for those who haven’t caught up all the way I’ll just give a little teaser — where Axe essentially is let out of a situation where he was in trouble, and he’s coming back to his pad, and it’s literally — you guys portray it visually as a completely raw bachelor pad — and the song was “Street Punk.”

      Brian: Vince Staples, yeah.

      Sonal: Oh my God, I fucking love that moment. It so stripped him bare, down to just he’s a street punk. Tell me about that decision and that choice.

      Brian: I mean, David and I choose all the music for the show together, and we’re both music fanatics and trade music all the time. And we put music in the scripts. So when we’re writing that script, we’re going back and forth about what it should be. Is it hip-hop? If it is, who is it and why? We had Vince Staples on the list since the end of the first season I think, when his first record came out. “Norf Norf” is what I thought we would use from the beginning. 

      But at that moment, you know — that moment people really understand what happens when Axe gets in that hot tub. And, again, that was in the script, that was what our goal was — and then we had to work incredibly hard with our brilliant editor who figured out how to make that sequence work the way we’d had it in our heads. Marnee Meyer, who edited that episode, really worked incredibly hard to build that sequence so that it matched and then exceeded what we had written. And Marnee’s been with the show from the very beginning — she and an editor named Naomi Geraghty have been with the show from the start, and are really and truly our creative partners. They’re the guardians of the tone of the show with us.

      Sonal: That’s great. All right, I’ll ask one last one and then we can wrap up. So, in season one — does this count as a spoiler alert because it’s so early in the season? I’ll just give it a high level.

      Brian: We’ll decide.

      Sonal: Okay. There’s a scene where you essentially set up Axe. The entire audience thinks that he’s gonna cheat on his wife, and I spent that entire episode on the edge of my seat worried that he was gonna cheat on his wife.

      Brian: This is an acceptable spoiler.

      Marc: This is a spoiler. This is totally a spoiler.

      Sonal: But it’s an acceptable one.

      Marc: 100%, I don’t know how you could conceivably think it isn’t.

      Sonal: It’s season one. Okay, fine, guys, but just quickly on that, like — that was obviously deliberate. Like, tell me about the decision making behind that.

      Brian: So, when I was saying the thing about sitting on the couch writing, and how that is this incredibly free process. Then you have to rewrite, and then you have to think about how it fits into the whole. So the whole gag is to write with total freedom, and then rewrite with total clarity. And so, when we’re thinking about whether a character will behave in way A or way B, we’re thinking about what they would do in the moment, and then we’re thinking about the ramifications of that. So, if the character did decision A, well, what does that then say about that character as we go through the rest of the series? Which will leave us in a place where there’s more optionality? And it’s clear in that case which one would leave us with more optionality.

      Sonal: That’s great. Okay.

      Brian: Oh, can I say one thing though? One of the great things about something like this is that, someone like Marc can do the work he does, and then I can do the work that I do, and if there’s some sort of a mutual sort of fascination with the work, you get to connect with people on that. And that is one of the, sort of, unintended joys of the work that I get to do. And so, that’s why I was happy to fly out here and do this podcast, because we’ve gotten to know each other over the last few years and it’s been a real pleasure. Thanks for having me here.

      Marc: Thank you, Brian.

      Sonal: Thank you so much for joining the “a16z Podcast,” Brian, and for coming out here. We really appreciate it, and “Billions” the next season is now out.

      Brian: March 17th.

      Sonal: Thanks, Brian.

      Brian: So happy to be here.

      Sonal: Thanks, guys.

      Marc: Thank you. And by the way, people may not know — I actually play on the show. I actually play Wags under a rubber mask, and so, that’s why you never see me in a cameo.

      Brian: I thought we weren’t supposed to advertise… 

      Sonal: Oh, my god. Wags is one of my favorite characters. Well, thank you…

      • Brian Koppelman

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Product-Market Sales Fit (What Comes First?)

      Jyoti Bansal, Peter Levine, Satish Talluri, and Sonal Chokshi

      One of the toughest challenges for founders — and especially technical founders who are used to focusing so much on product features over sales — is striking “product-market fit”. The concept can be defined many ways, but the simple definition shared in this episode is: it’s when you understand the business value of your product.

      And that comes down to users, which is where the concept of “product-market-sales fit” comes in, observes Jyoti Bansal, founding CEO of AppDynamics (which was acquired by Cisco for $3.7B the night before it was to IPO). Bansal shares this and other key milestones and frameworks for company building in conversation with a16z general partner Peter Levine; enterprise deal team partner Satish Talluri (who was a director of product and growth operations there); and Sonal Chokshi.

      So in that shift from product-market fit to product-market-SALES fit, how much should you optimize your go-to-market for product… and even the other way around? What does this mean for product design and product management? When should companies offer services? As for pricing, how do you know you’re not leaving value on the table? Again, it comes down to product-market fit: If your business case is strong, you will not be leaving money on the table, argues Bansal in this special podcast series on founder stories and lessons learned in enterprise go-to-market.

      Show Notes

      • Discussion of product-market-sales fit, and developing products that can be sold [1:26]
      • How product and marketing strategy need to be aligned [6:52]
      • When a product has no existing market [8:55], and the importance of ensuring engineers understand the sales process [11:54]
      • AppDynamics’ approach to sales (top-down vs. bottom-up) [14:36]
      • Discussion of company building [19:33] and selling new products [24:28]
      • Prioritizing products, managing growth [31:15], and pricing and packaging [35:00]
      • Advice for assigning roles and responsibilities [45:28]

      Transcript

      Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today’s episode continues our enterprise go-to-market podcast series. And the theme of this episode is for founders and product managers to consider the tight relationship between product and go-to-market, one informing the other in both directions. What are the key milestones that go into both, and in different phases of company building — especially pre to post product-market fit? The conversation features special guests Jyoti Bansal, founder and founding CEO of AppDynamics. He’s also a co-founder at Unusual Ventures and co-founder of Harness. Joining me to interview Bansal, we have general partner Peter Levine, who also put out a series of 16 short sales videos for founders, which you can find at a16z.com/16sales. And then we have a16z enterprise deal team partner Satish Talluri, since he too came from AppDynamics, where he was last a senior director of product and growth operations pre-sale to post-sale.

      Speaking of, we go beyond the typical discussion of product-market fit into the concept of product-market-sales fit, and what that means for product design, to services, to pricing and packaging, to product management, and more. But first, we quickly began with the fundamental shift in mindset for technical founders. The first voice you’ll hear is Jyoti’s, followed by Satish’s, talking about the initial insight behind AppDynamics, which was acquired by Cisco last year for $3.7 billion, the night before it was set to go public.

      Product-market-sales fit

      Jyoti: I was working as an engineer in a company. And this was before the phrase you guys coined, on “software is eating the world.” This was 2008, right? But it was clear that software is eating the world, right? And to me, it’s like, okay, if everything is going to be software, something goes wrong in software, someone needs good tools to troubleshoot and fix it. So that was really the insight. AppDynamics was building, monitoring and troubleshooting solutions for complex software apps. So if you are on online banking and something goes wrong in your online banking, you will use AppDynamics to figure out the root cause and fix it. Or if Delta reservation systems are down and everyone is stuck in the airport, someone needs to find tools to troubleshoot what’s the root cause of the problem and fix it. 

      That’s what AppDynamics built, those troubleshooting tools. So now it’s like, when I jumped into it, I didn’t know anything. I didn’t know how to raise capital. I didn’t recruit anyone before AppDynamics. So you had to go and figure that out. I didn’t know how to have lots of customer conversations, or even find customers to talk to.

      Satish: At least during the pre-product-market fit, a lot of engineers — even including myself — we get obsessed with the technology and not so much about the user. At the end of the day, if the user adoption is not there, it’s no good.

      Sonal: I mean, there’s no market without the user.

      Satish: Exactly.

      Sonal: You got the product side and not the market.

      Satish: Exactly. I feel a lot of engineers, to be honest — struggling about understanding the customer and user adoption and the engagement metrics without that good UI, UX. And really, [whether] it be the open source strategy or the closed door strategy — doesn’t matter — but user adoption is what should be driving the pre-product-market fit.

      Jyoti: Now the challenges completely change after you have your initial product-market fit. They become all about sales and learning sales and scaling sales. And it’s almost like the companies go through that journey, right? The pre-product-market fit, the challenges are different. Then after pre-product-market fit, the challenge becomes about selling and scaling sales organizations.

      Sonal: You’re saying on one hand that you have to sell after product-market fit. But on the other hand, I’ve heard that for a lot of enterprise businesses, part of the act of selling is finding those users in the first place. It’s a bit of a chicken-egg thing.

      Peter: Well, a lot of us start our careers as engineers, and a lot of our construction of a business is around the features, and around what the product does. It’s all technically oriented, right? Because what we often say is, “Okay, well, if we add these features, then people will come and buy it,” and I find that some of the go-to-market is an afterthought. Once you’ve built something, and I would argue, in today’s day and age — if you’re going after small businesses versus large enterprises, or self-serve, or whatever — thinking about that up front, along with the product requirements and technical requirements, may be a good thing to go and do. Like, I think, to sequentially order those probably results in an efficiency issue, right? We go build something and, oh, like, who knows how to go sell this and all of that. Might it be useful to say to technical entrepreneurs, “In order to do this, you got to go figure out the go-to-market as well as the product features, and don’t eliminate that or push that off?”

      Jyoti: I would totally agree. The way I mentally think of this is, two phases of product-market fit. The phase one is really even figuring out where your target market is. So for that one, you really want to start broad and then segment. Like, if you don’t know <Yeah.> where would your idea or your product fit the most — is it large enterprise, is it SMB, is it financial services — then I would just go and interview all of them, and not narrow yet. And start building the product, which is a little bit wider.

      Peter: And how did you guys come to that? Where did you start? You had this wide aperture, and then it narrowed. So what was the first thing?

      Jyoti: To me, it was that people are building these complex software apps, and they need to monitor them well. <Sure.> And I had the technology idea that if you can instrument the code and trace everything, then it will be a good product. But I didn’t know who would buy it. I started like, okay, let me go and broaden it. Let me go and find people in larger enterprises to talk to, let me go and find people in startups to talk to, let me go and find people in midsize companies to talk to, and see where it sticks the most — of where the most pain is. 

      And what I found was, okay, the most pain is where there are these, kind of, medium to large companies, which are building these complex distributed Java applications. So let me now focus more on that. So I started to broaden, then we started narrowing down a bit of the focus and — but after that, once you identified, then you — it’s very important that you marry the go-to-market model in your product thinking. Because these days it’s all very tightly coupled together. You don’t have, like, sales is different, then marketing is different, then product is different, then all — it’s all together in many ways, right?

      So if you have an open source model, or you have a freemium model, or if you have a — is it SaaS, is it on premise, is it hybrid offered, is it going to be land and expand — and you have to engineer a product with that in mind.

      Peter: Right. The features of the product almost have to inherit part of the go-to-market within the product itself, right? And a lot of product design, I think, reflects the go-to-market attributes that need to be considered.

      Jyoti: So in AppDynamics, I used to say, like, it’s a little bit misleading to just call it product-market fit. We should call it product-market-sales fit.

      Sonal: Oh, I love that.

      Jyoti: It’s like, have we found the right…

      Peter: Oh, that’s a good one.

      Jyoti: There’s a right market, and you have the right product, and we have the right sales or go-to-market strategy that works for it.

      Sonal: When you said two phases of product-market fit — the first one was where — like, either the small, big enterprise, different domain or industry — and the second one was the sales motion?

      Jyoti: So, the thing of phase one is, like, where is the most pain — and where your product or your unique approach, or whatever it is, solves the pain in a way that people will pay for. And you’re also validating, like, your technology — does it really work? Can you really build the product? Does it really solve the pain? And then you have to figure out, like, what is the sales strategy or go-to-market strategy that will work and scale. And does your product support that? Because if your product doesn’t support it — you know, many times people are, like, we’re gonna build a freemium strategy. But the problem is, if a product is too complex, freemium doesn’t work.

      Peter: But just to drive this point home, which I completely agree with — the product — the features of the product need to inherit part of the sales motion itself, right? And that if you’re going after a certain motion, or a certain customer, the product needs to be reflective of that. And I think we often miss there. Like, we build a product, and even if we define a go-to-market, the product features or the interface, the design — may be completely misaligned with the target audience, or target go-to-market, I should say.

      Jyoti: And some of it you can also break into, say, revenue goals. Like, I would roughly think getting to your zero to the first million ARR — you are in that phase one of product-market fit. Like if you have a product…

      Sonal: ARR, as in the annual recurring revenue.

      Jyoti: The annual recurring revenue, which is, like, do you have a product someone will buy and it’s solving something? Then you’re like, a million to the $10 million in revenue — that’s where you’re iterating on the go-to-market strategy and getting the product to be aligned with that. And if you get that right — that, like, at $10 million, you should be there. Like, you got the product-market and sales fit as well. <Yeah, yeah.> And then you can press, you know, the gas — and go from $10 million to $100 million from there. But you got to get that iteration on the sales fit to it.

      Products without an existing market

      Sonal: So I have a question for you here. So in your case, you had a product where you knew the tool was solving an existing problem. Does that calculus change if you’re creating a category, and you’re going into a market where, “The problem does not already exist?” Because then you don’t actually have the ability to necessarily know where or how to figure out the sales motion yet — or is that not true? Because I think a lot of founders might argue that, “Well, why can’t I be like Steve Jobs and sort of invent — like, create the product that people all go to.” Like, what would you say that?

      Jyoti: Well, you’re always creating — you know, either you’re solving a problem significantly better than others have done in the past. And that the dimension of what does significantly mean could be different. It could be you’re 10X more scalable, you’re 10X more easier, you’re 10X more cheaper — whatever it is, right?

      Sonal: Right. All 10X better.

      Jyoti: It has to be 10X better in some dimension. So in an existing problem, or if it’s a new problem that’s emerging — then it’s, you know — you still have — the problem has to be there. Either there’s a problem with existing vendors, or there’s a problem because there is no solution there. But it has to be there. Otherwise, you don’t have anything to sell.

      Peter: And I think in enterprise more than consumer, there’s a budget — there’s a certain budget dollar that you’re gonna go after in enterprise. Maybe it comes out of the development budget, it comes out of engineering, marketing, sales. There’s something for which you can at least start to frame this new thing, new market, whatever. Like, one of the questions to ask is, “Who’s the potential buyer for this?” Even if it’s a totally new market, right? But let’s call it an enterprise product somewhere — doesn’t exist before. Still, who’s the buyer? And what budget does it come out of? And a lot of products, actually, where there isn’t a market yet, may span multiple buyers, in fact — may come from multiple different departments and span budgets. And you need to think about that, like, okay — I’m creating this new market, but perhaps the buying motion and what the customer is used to actually doing, from a buying behavior, is so complicated, it’s never gonna happen.

      Satish: Yeah. Even for the product managers, or the founders, it always helps to do a sales kind of play. Wherein, how exactly you’re going to sell, and who is the actual buyer? And who’s the actual user? What are you going to say to the user, what pain points you’re going to solve. And what exactly — how exactly the user is going to use your product. 

      I think, working with the salespeople who are actually on the frontlines to go and sell — for the engineers and the product managers, it really helps. And in fact, at your company, you made a lot of engineers to go on those calls, to literally understand — who exactly is buying that product? How much is he going to pay? And for that, what exactly you need to build. So that connection for the engineers — to go on those sales calls — really help them to understand that sales motion, and how to incorporate [it] into that product. That’s one of the best practices I love. So that’s how engineers always got to understand that sales motion.

      Jyoti: We had that strong belief is — just that we have to break the barriers between engineers and customers. In the startups I’ve worked at before AppDynamics, as an engineer, people will say, “Engineers don’t know how to talk to customers, so let’s keep them away from customers.” And so we’re selling to engineers — like, our products are technical and, like, you know — so that just doesn’t make any sense to me.

      In the early days of finding the business case — the finding, like, you know — where the budget will come from — one of the questions that we always ask — like, my favorite question to ask to any customer was, “How would you make the business case to your boss to buy this?” And that’s when you would start hearing like, this is my business case. Like, every time we have an outage, we’re normally spending six engineers in a room for five hours to try to figure this out. Now, with you guys, I can reduce it down to one engineer for 15 minutes. And that’s my business case. And once you start hearing the business case, then you can know that there is a business case — you can monetize it, and you can convert into dollars at some point, right?

      Sonal: How do you navigate that, though, when you have multiple budgets and multiple decision makers inside the enterprise? Different groups or departments have different problems or itches that you’re scratching. And how do you sort of up-level it, so that you’re selling into getting the big bucks — and not just, sort of, the incremental budget?

      Peter: I mean, I would say it all depends, again, on this product-market-sales strategy. A lot of companies that start with bottoms-up only go after an individual user — and then they get enough use on individual users and propagation from the bottoms-up. I call that self-serve. So there’s no complexity there. There’s no multiple buying centers or whatever.

      Sonal: Right. You just decide.

      Peter: So it’s not a foregone conclusion you just — you know, that that is the way to go. Now, after enough people are using the product, then you can come in with tops-down and say, “Hey, did you know like everyone in your organization is already using the product? You ought to have a corporate-wide license so we can private support and all of that.” So that would be an example of bottoms-up and then coming in on tops-down. 

      Many other products, though, are designed to be tops-down as a starting point, because it may go across departments, it may be more complicated, <Security needs.> there’s security needs, whatever — where a bottoms-up design just doesn’t work, right? In which case, then, you probably need to start with a more traditional — I’ll say a direct sales organization. It could be an inside sales organization or direct, calling in and actually getting the customer. I mean, complex sales often requires multiple buyers, multiple parts of the organization to come together. And that’s a skill set that a well-honed sales organization will know how to do.

      Top-down vs. bottom-up sales

      Sonal: How did you guys — what was your sales motion at AppDynamics?

      Jyoti: So ours was a combination of both. You know, at AppDynamics we call it the Sandwich Strategy. You go from the bottom, you go from the top, you will go to the developers and DevOps engineers directly. It was done through a freemium kind of model, so that they will start for free, and they can use a light version of a product for free. And then we will start going from the top, where we’ll create air cover and, like — when we have multiple users in an organization, then we’ll go and sell them more. So really, the sales motion was built on — the end users can start for free, then we’ll have — sell them, like, some license of it. We call it “land and expand.” The land deals, which are like, you know — say, $20,000, $30,000, $50,000 deals. On [the] phone, we can sell that. And then we’ll expand into, like, half a million, million dollar, $2 million. You know — now, these days, $10 million deals. So that — you’ll need traditional enterprise salespeople where you will do that.

      Sonal: Right. So inside sales versus field sales, basically, in that context. No? Not right?

      Jyoti: Yes, but in most of these companies today, you would probably need both. <Okay.> So it, like, depends on, like — if the model is only top-down, you probably need only field sales, and you’re selling into large enterprises. But if a model is this kind of a “land and expand,” you want to do the land through fee inside sales, and you want to do the large — the expand into — [with] field sales.

      Satish: Yeah. And of late, we are seeing scenarios in which once you go to the top — and if they are big enterprises, services has become a very important component of it. To be honest, a bottoms-up developer option, great land. But once you expand, and once you get into multiple product portfolios, and into complex integration, services is [an] essential component of the enterprise sales.

      Sonal: So tell me the takeaway on services, because I’ve always heard — and disillusion me if this is not correct — but services are the things that reduce your margin. So you don’t want to have too many services. Or, what’s — how do you balance that one?

      Jyoti: “It reduces the margin” is from the perspective of you as a vendor. But think from the perspective of a customer — like, if they spent a million dollars on your product, and they’re not getting the value of a million dollars, because they didn’t have enough — the right people in place to implement your product — that’s not good for them. And eventually is not good for you, because you’re building a — likely a recurring revenue business of some kind, right? When we started, we were like, we’re not gonna sell services — not from a margin perspective — because we wanted our product to be easy enough that no one needs any services. And that was true for a long period. For the — actually, for the first four years, we had zero services. And then we started getting into larger and larger enterprises, and larger and larger deals, where people were spending millions of dollars with us.

      Sonal: Yeah, you want to save that money.

      Jyoti: Yes. And we figured out like — if they don’t buy any services, sometimes no fault of our product — they just don’t get that option that we want. And then we were like, yes, so — like, too much services, then the margins are low, right? But we’ve found the right balance was about 10% to 15%. So like, if in our products — if, like, people are buying — let’s say, they’re spending a million dollars with us on the software, and they spend, like, $100,000 or 10% to 15% on it on services, their adoption is much better and much faster.

      Sonal: So ideally, you actually make more money on the upsells and cross-sells and more feature expansion, based off that 10% to 15%?

      Jyoti: Eventually if your users are getting adoption and [are] happy with the product, the money will come. The margins will come, right? So you have to figure out, like, people are getting value or adoption or not.

      Sonal: Margins will come, I like that phrase.

      Peter: If you think about, in that example — let’s say services, in that case, leads the buyer to purchase a million dollars of license, the blended margin on that is extremely high. Much higher than it would be on a $20,000 no-services deal, right? <Right.> So while services from a unit economics standpoint may be, you know, a little more expensive from a margin standpoint, if it drives very large deals <You come out ahead.> with software margins, you come out way ahead. So you have to think about blended margin, and the idea that services are often a leader into a company buying the million dollar, $2 million license. It’s just expected as part of that.

      Jyoti: And the renewals. Like, the year two, year three, year four renewal offer, right? <Right.> So if you spend — if for the first year, because you sold services, your margin may be lower — but because of services, the adoption is higher. So your chances of renewing in year two, year three, year four, year five are much higher. So the margins for those will go up.

      Satish: At the end of the day, adoption is what counts for a product, right? And services help. And also, keeping aside the financial aspect, even from a product aspect, it’s good in the sense that we hate shelfware. What good is it if some enterprise bought $1 million, and if they’re not using it?

      Sonal: Just sitting on the shelf, right.

      Satish: It’s really bad. From a product standpoint, there are lots of these minor features which are custom. They don’t fit in the product, they actually fit well for the services. So that’s why having a good combination of what’s going into the product versus what should be left in the services — that’s a good play for the product manager or the CEO to make that call. So that the product adoption goes well. Adoption and the product — services is a necessary complement after sometime.

      Company building

      Sonal: It goes hand-in-hand. That’s right, I’m really glad you brought that up, because I want to segue to talking about the company building side of this. So, you’re describing the sales motion to customers — and the product-market fit pre-product-market, pre-product-market-sales fit, and post. Let’s spend the rest of the time connecting it back to what happens inside the company. So you’re describing the product — how does this affect the product roadmap? Like, when you get all this feedback from customers, and you have the sales motion in place, how does this then drive back inside your company to further developing more features on a product — making those balancing decisions for what goes into the core, to what goes into the custom, to what goes into the next iteration? Kind of, tell us about those trade-offs.

      Jyoti: It depends on different stages of the company. When you are in the very, very early stage of building the v1 product, you really want to use the customer feedback to figure out what you want to build that will sell — that will get your first 10 customers, first 20 customers or so. And, you have to listen to the customers. That’s the product-market fit exercise, the customer validation exercise and all that, right?

      Sonal: Are they paying for this thing, too.

      Jyoti: Yes. And once you have customers, and then you, like — how you prioritize becomes what you’re hearing from customers, what will it take them to be successful and adopt the product more and buy the product more. And you want to make sure that the product team’s ears are open — listening to customers, listening to customer support, customer success — they are watching the tickets, they are watching, like what’s working, what’s not working. Then sales is trying to expand and get more customers. So you have to work with them as well, because [of] your competitive pressures. You have to, like, catch up to competitors on some features sometimes. 

      And so you have to make sure that you’re winning enough in the market, you can get enough revenue, and you prioritize that also. But then there’s a third part, which is, like — you also want to keep expanding your product, which are things that your current customers are not asking for, but you need them for expanding your addressable market for customers, right? And that’s where it’s — from a product perspective, it’s a balance of those three things, right? It’s the — what do we need to win more revenue today? What do we need to keep our customers happy? And what do we need to win more revenue two years from now?

      Sonal: So win and keep now, to what do you need in the future to win.

      Jyoti: Yes. And the rule of thumb that I followed — that was [that] two-third of our engineering investment should go with our existing TAM. <The core base.> And the one-third of our engineering investment, we should keep putting on expanding our TAM always. So our total addressable market, right? So when we started with, like, let’s say our initial v1 product was application monitoring for Java applications. And that was our TAM. Once we had that — like, we’ll start putting one-third of our engineering on expanding it to the next addressable market, which is application monitoring for .NET applications. After a year, that became part of the .NET — our product became Java and .NET. Now we look, okay — what is the next addressable market where I can put another one-third of my engineering. And then we kept doing it systematically for seven, eight years. And we just kept expanding our TAM.

      Sonal: So the two-third, one-third rule.

      Satish: Yeah, but the interesting aspect, even in — during that expansion, is that the target buyer — because we had an existing sales motion, target buyer, and user — we didn’t change that drastically, because the sales motion is already oriented towards it. So it’s like those agencies — be it .NET, or the end user monitoring, and so on, so forth — still, it’s targeting the same buyer and user, so that you can leverage your existing go-to-market sales motion. That didn’t cause too much of — distractions on the go-to-market side. That really helps expand your product portfolio, but at the same time, leverage your existing sales motion to go and attack and expand the market. So understanding that if you change both product and also your sales motion, suddenly — then it’s almost like, again, building from scratch, and that causes lots of disruptions in the company.

      Peter: That’s exactly right. I mean, if I go back to the sales videos that I did, there was a concept in there called the sales learning curve, which says that at different stages of building out a sales organization, there’s different people you need. When a new product comes out inside a company, you often need to start a new sales learning curve. It’s not just the old one that you follow, <Mm, interesting.> but you may have a new customer, it may be a new market motion, whatever. 

      And the old organization may not be — because they’re at a mature level of selling an existing product, and now you start out with a different product. You may have to have the evangelist salesperson start that new sales motion, and not have the bigger sales organization take on that product in a new market. They might not be able to do it, and a lot of companies fail at that — because they assume just because they have scaled with one product line, that they can introduce another one — let’s say, for a completely different market in there — and nothing happens, right?

      Jyoti: And that — we learned that at AppDynamics the hard way.

      Sonal: Tell us why, how it was the hard way.

      Peter: Yeah. A lot of companies do.

      Jyoti: Yeah, because we built our first product, and it was selling, and the sales process was mature. We were a mature sales organization. Now we started building our second product. And, sort of — you have a mature sales organization, [so] let’s give them the second product to sell. They started selling the second product and they failed at it. And I was, like, these people are so good [at] selling — they’ve been so successful at it. But the challenge is, like, when you have 100 customers and you have like 50 references and you have — like, you are in the Magic Quadrant for something, everything is well refined. And how you sell is different than when you’re a brand new product with zero customers. So they just started struggling. 

      And they say your product sucks, and this new product is not good, so let’s just throw this away. We should not build any new product. And I was, like, if you’re not going to build new products, our growth will slow down. So we have to learn how to make [it] and sell it. So we internally structured as like — if we were good at selling a new product, so what changed? So maybe we should build a model which uses the same thing that worked for the very first product. So we reorganized ourselves in a model — like, all startups within a startup. So, like, we are a startup, but we’ll form new startups inside it. And we’ll sell the same way — the way we sold our first product in the beginning.

      Peter: Right. This is a well known problem. Often the second product never takes off because it’s — it doesn’t get the visibility or attention or expertise that’s required when a new product is released.

      Jyoti: And there’s a sales compensation aspect to it also. Because the salespeople, they can sell the mature product — which is easy to sell at that point and make their numbers by doing it. Now you give them something — a new product, which is much harder to sell.

      Sonal: Because you’re not gonna make your numbers. So how did you adjust the compensation accordingly?

      Jyoti: So you almost have to create a separate, almost evangelical new sales team whose job is to do that, like the way you…

      Peter: Yeah, just like you did in the beginning. Just like you start out. You don’t even know what the productivity is, you don’t know a lot of things. And you learn that across that new product line, just like you would do at the start of a company.

      Sonal: So this connects the dots between the idea of a startup within a startup, the evangelicals or evangelism that you mentioned, and a different sales learning curve for each — they’re kind of all the same thing. That makes a lot of sense. I mean, quite frankly, the analogy that came to mind for me, as an ex-developmental psychologist — is that when you’re doing some kind of a research study, you can never know the effect of variable X if you’re manipulating too many variables at the same time. What you’re really describing is isolating one variable in order to diagnose what problem — so you can then sell, in this case, that is the solution.

      Peter: However, it’s expensive to go do that. To have a startup within a startup — like, here I have my sales organization, now I have to go hire more salespeople that are different than the ones I already have, to go sell this new thing. And at what point do you say, okay — we have to — for every new product, do you have a new salesforce?

      Sonal: Well, what’s the answer? I’m asking you guys. <laughter>

      Jyoti: Well, I’ll tell you what we did. So what I brought into AppDynamics was that we said the sales learning curve has three phases. One is the — my first 25 customers for the product. 25, that’s like phase 1. Which is very, very — almost the founders are selling…

      Peter: We call that the initiation phase. Go ahead.

      Jyoti: Yes. There’s the 25 to 100 customers. And then there’s 100 — after 100 customers, a mature product, we can — our salesforce can sell it.

      Peter: Yes, that’s the execution phase.

      Jyoti: So the first 25 customers for the new products, we actually got the product management team <Exactly, yeah.>  to really sell it the way your founders will sell in a brand new startup — instead of hiring a new salesforce for that. But after that, our salesforce could take it. The phase two, they could take it. The same with phase three — they were good at it anyways.

      Satish: So for our fourth product line, which we call Real Time Business Monitoring, this is exactly what we observed. The existing salesforce was, like, more tuned to selling the existing product because it was a well trodden path. For the fourth one, we literally constituted what we call a SWAT team. It constituted the product management, a couple of engineers, the best sales engineers, and one solution architect. We literally went and sold some of the top deals and created the sales enablement material, the market positioning. And, in fact, once we created that, then we used it to train the rest of the salesforce — even not everyone. And once we hit that first 10 salespeople, they are cracking it. They’re making more money with better incentives. Then the rest of the salesforce is like, “Oh, there is something big there that I also need to sell.” So we had to do it in stages. But we literally did a SWAT motion for, like, eight months.

      Peter: It’s kind of like a flywheel, you have to get it going. And once it has some momentum behind it, everyone picks up on it. I would also say that, as the regular sales organization starts to sell this new product, as managers, you want to make a big deal about it, right? You want to, like, promote it and say, “Hey, did you know in the east region, we just sold new product X, and it was for $150,000, or a million dollars” or whatever. And that gets everyone, like, excited — especially if it comes from the CEO. Salespeople — I mean, everyone loves to be recognized for their success. And if it’s important to the company, then doing something as simple as that, from a leadership standpoint, also has a very beneficial upside for all the other people who want to get that recognition. It’s an easy thing to do, but you often may forget about it or whatever, as a CEO.

      Satish: Yeah, literally what Peter said — once we did this SWAT motion and created those initial amazing sales — the big dollar sales, upwards of million, and so on — literally, we did internal sales. We had to sell to the rest of the salespeople. We got our CRO, our CEO — they literally are the brand ambassadors of this new product and say, the message is simple. The best new product — you can sell higher, more, in a short time, and make more money. Now, for our annual sales kickoff, that’s the big message. And we got our customers in there, and we got the best selling sales reps, and the rest of the sales team sees and…

      Sonal: They want to get on board.

      Satish: Exactly, I want to get onto the train.

      Peter: Yeah, one of the things that I have seen on the, sort of, down — the negative side of this, is companies release too many products. And so then, every week, a new product manager is out there trying to promote.<Rally the team.> They’ll rally the team around this. So it’s very important to make sure you’re focused on a few things that are really gonna work well. And don’t let it, from a leadership standpoint, get out of control. Like, it’s great for people to try experiments and all that. But don’t let it get mainstream until you know it’s gonna be mainstream. Otherwise, there’s 50 products on the price list, and everyone’s fighting for visibility and it becomes…

      Jyoti: And it’s very distracting also…

      Peter: Very distracting.

      Jyoti: The salesforce is expensive. So if you take your salesforce that’s doing — and you try to give them too many immature products to sell, you’re reducing their productivity, your expense goes up, it’s not good. So you do want to get to that level of maturity before you give it out to your broader salesforce.

      Prioritizing and managing growth

      Sonal: Right. So tell me, though, as the leader of the company then — because you have these processes inside, I’m hearing the broader context of the trade-offs of both approaches — how did you strike the balance and figure out what to focus on, and then what to sort of keep off the list? I mean, you have a lot of interesting rules of thumb so far. Steve Jobs, in the Walter Isaacson biography, made the entire team list all the best things that they were learning that they could do — and then they crossed everything else off the list, except for the first top three. Like, what was your process for that?

      Jyoti: Our process, I would say, as a startup — most of it in our case came from that two-third, one-third rule. Like, one-third of our engineering investment, we can put on expanding our addressable market. The two-third we put on serving our currently addressable market — which is, like, improving the product, adding features, capabilities, all of that. And one-third, we improve on, like, new use cases, new addressable markets, new addressable users that we are currently not serving. So whatever will fit into that will define that. Another system that we use is working backwards from a longer-term goal. We put in this plan called our path to $100 million revenue. And when we say, okay — we want to get to $100 million revenue. And how would our business look like? And how much we can do at a fast pace without our existing products, and what we will need to add to the new products to get there.

      And then we also have, sort of, a rough timeline with it. But once we got there, we put a new plan together, which is our path to a billion dollars of revenue. So it was like — okay, from $100 million to, if you want to get to a billion dollars of revenue, what would our business look like? And we realized, like, when we did the math — and this is, again, a rough math. Like, you never know. That if we want to get to a billion dollars of revenue from $100 million in, like, 7 years, let’s say — or 6 years, our plan was — we need to have, like, at least 40% of our revenue coming from these new adjacent products. Otherwise, our growth would not get there. And then we have to build this. So there was the part of, like, what you can do from a bottom-up investment perspective. Like, engineering the sources and what you need top-down to get to a billion dollar revenue goal.

      Sonal: Right. It’s working backward to provide the focus.

      Jyoti: And you need to do both. <Right.> Define the intersection of, like, what you can do bottom-up. You can’t build 10 new products, so you — what you can build — and then what you need to build to get to some kind of revenue long-term goal you have.

      Peter: And at that point in time is probably when you start to have an M&A function in the company, to start looking at, outside…

      Sonal: Why?

      Peter: Well, you have organic growth, which is using your team to go build whatever needs to be built. And then you have inorganic growth, which is basically buying or licensing technology in teams that are not inside the company. Because I would argue if AppDynamics was growing to be a billion dollar company, and you had the capacity to support, from an engineering standpoint, $200 million or $300 million of product design — how are you gonna build 10 new products, if that was the envelope that you’re — or even three products, right? So at that point in time, it’s a build versus buy. Do you raise more money and go build a team to go build something? Or do you go buy something and integrate it in? And both have their challenges, but that’s another function inside the company that usually comes about that point in time.

      Jyoti: Like, we acquired three small companies. You know, we did that for different things, but sometimes it’s like just accelerating the time to…

      Sonal: As I just say — it seems like it comes down to speed to market and what the competitors are doing.

      Jyoti: The speed. Yes. Like, if you build from scratch, like, from zero lines of code. It would have taken us two to three years to get a reasonable product in the market. By the time we matured it, and found the product-market-sales fit of it, and all that. <Yeah, for sure.> But if we acquired something, we could probably cut down from two or three years to half of it, right? So maybe, like, maybe even 75%, in some cases. So that’s always a factor.

      Pricing and packaging

      Sonal: Okay, so why don’t we then just talk a little bit about pricing and packaging, because that’s such an interesting subset of this. So we’ve so far talked about the product-market-sales fit, the go-to-market and the product as a part of that, obviously — because they’re the two things you need. How does pricing and packaging come into this? Because that’s a really top of mind question for a lot of founders.

      Jyoti: Pricing and packaging is a complex thing. Pricing is probably more complex than packaging, in some ways. I look at pricing as more a function of, what is the business value? If someone buys your product, is it worth $50,000? Is it worth $100,000? Is it worth $300,000? How would people justify? So that’s definitely one function. Second is, like — the rule that I’ve used for pricing is, can your salespeople describe it simply? Like, a customer is going to ask a simple question. “How do you price your product?” And if you can’t describe it in half a sentence, you have too complex of a pricing. 

      And yes, there could be, like, nuances to it, and there could be, like, details to it, but you have to be able to describe. Like, in AppDynamics, we’re monitoring all kinds of different systems, right? So the pricing was complicated. But we said, okay, the simple pricing philosophy that our salespeople can tell us — we price by how many production systems you have. And that was kind of the rough unit of pricing. And we can measure production systems in different ways, but that’s how we price it, right? And that’s at least simpler, that your pricing philosophy is simpler for people. So that was my rule number one.

      Rule number two, there was — that whatever the pricing is measurable, because if it’s not measurable, and now the customer says, “Okay, how many licenses do I need to buy?” Our salespeople cannot even tell them very clearly like, okay — this is how you measure how many you need to buy. However, it will create a lot of friction. Or like once they buy it, we can’t measure and track, like, how many are using it. That’s a problem as well, right? So if we can describe our pricing in half a sentence, that’s one. Second is that it’s measurable, that people can measure presale, and people can measure post sale — we have a good pricing system. The question after that is, okay — what is the price, like the dollar price of it on — for that model per license, how much you pay? That really to me — it comes down to business value. And enterprise software, especially selling to large enterprises, I argue to most founders that you should price more than you think you should.

      Sonal: So we always say raise prices. That’s our mantra around here.

      Jyoti: Exactly. So, price higher. You can always discount. If there’s not value, you can always discount. And customers are not gonna pay more than what the thing they should pay anyways. So you can always discount and then go there, instead of pricing low.

      Peter: I love the idea of this pricing framework. A lot of companies try to come up with a new model of pricing. So instead of price per user, or price per application, it’s price for the number of, you know, air vents your server has, right — let’s just say. You have to come up with pricing that the customer is used to actually paying for. If you start to create something that’s totally new, it creates friction in the system. So it’s typically users or capacity, or numbers of something. I love the measurable piece. A lot of companies try to get overly cute, and it gets overly complicated and then salespeople can’t explain it. And even if it’s simple, if it’s not understandable by the buyer, they’re going to be like, “Well, what does that mean?”

      Sonal: So I’m hearing you say [to] founders out there, be creative with your product, but don’t get creative with pricing. Like, do what you need to do that makes sense to the buyers.

      Peter: Exactly.

      Satish: This difference between consumer purchase versus enterprise — enterprises, they need a little bit more certainty. You’re getting it from a budget, right? They’re already pre-planned. That’s one. <Right.> And they want certainty, in the sense that — oh, is it one alert or [a] thousand alerts? And if it’s too variable, then suddenly if it blows the budget, [they] cannot manage it. So that’s why they want that certainty and visibility into that price.

      Sonal: So by certainty, you mean they don’t want surprises.

      Satish: Exactly.

      Peter: But you need to be able to be reasonably predictable on this — to not have surprises at the end. To say, “Well, it’s free, and then we’ll measure it in the future,” but they don’t know how to budget for it.

      Sonal: But how do you, as a founder, know that you’re not leaving value on the table, when you’re giving that certainty — or, like, surety that this is what you’re gonna get?

      Jyoti: If you put in, like, the — a good ROI process in your — as part of your sales process, that’s how you guarantee you’re not leaving money on the table. We have a very structured sales process. And in the sales process, we would look at, like, you know — what is your current state of doing this? How much is it costing you roughly, let’s say? What would be a new state with AppDynamics, and what would that cost you, and how much money are you going to save? And then price is a little bit of a function of — how much money are you going to save, and what’s your ROI? And that’s — if we are charging more than what it’s gonna save them, they’re not gonna pay for it anyways, right? Most companies, I see the mistake of, like — especially selling into large enterprise, and you’re asking for half a million dollars [from] someone — if you don’t back it up by a business case, people are not going to pay you. <Of course not.> And then you leave a lot of money on the table. If your business case is strong, you will not leave money on the table.

      Satish: Yeah, we had a process called Business Value Assessment, BVA. It went along with the sales process in which we always had that premium positioning. And we enabled our sales to convey why we are premium. And, secondly, we had those steering committee meetings in which the big check pair — we literally read out the ROI value use cases back to them, so that they can justify internally as to why they’re paying that premium. So giving that message, so that they can repeat internally and justify it — that’s what helps the part of the process and the premium that we can extract from it.

      Sonal: So what I’m hearing you say is that sales enablement created — it smoothed the road, kind of greased the wheels for you. But then on top of it, you played back and made sure to play back the ROI, so that then your internal champion could continue advocating that it has value and keep moving that forward.

      Jyoti: And justify that premium pricing.

      Sonal: One question I have is, the third element you mentioned, Jyoti, in your framework — the value — that’s the big kind of gray, goosey area, because that’s the least measurable one. How do you know the value to the customer? Did you just say, like, the simple —what the opportunity cost [is] if their systems went down? Or did you think bigger than that? How did you figure that out?

      Jyoti: The best way is to ask the customers. And this is something I would do in the product-market fit phase. A lot of people say what is product-market fit, even the initial one?

      Sonal: It’s a very philosophical question. What is product-market fit?

      Jyoti: Yeah. And such a vague question. My simple definition is, if you understand the business value of your product, that’s — then you know. And so, you know, the question that I used to ask in the product-market fit phase was, “How would you justify the business case to your boss?” So, like, I’m talking to a, say, a director of DevOps. I’ll say, “Okay, well, how would you make the business case to your boss, if you have to buy AppDynamics?” And they said, “We have one outage a month. Every time we have an outage, we put six engineers there, and this is how much it costs us. And because our users have a bad experience, we lose this revenue. This is how much it costs us.” 

      And once I start hearing it, I know, like — this is what I would — I would like to teach our salesforce how to make the business case, because this is how the customers are articulating. And unless you understand the business case, you don’t really have a product-market fit because that’s what you have to engineer your product around.

      Sonal: Right. So you’re saying the value is defined by the customers. Do you guys have any thoughts on how to define the value? That sort of loosey-goosey, vague thing of — you want to make sure you’re selling value?

      Peter: There’s a couple of things, which I’ve used in companies that I’ve run, is — if you look at competitive products, what are they? What’s the chart? How much do those costs? There’s an overall stack of technology. And if you’re providing a certain solution, what is that stack, in general? How do people — how have they budgeted for that? And then I always like this concept of “charge more than you think” in there. And you can always discount back to make sure that you’re not really leaving value on the table. I didn’t say money — I said value on the table, which I think is very important. 

      The thing that is — also I learned along the way, is — customers actually like to spend money for value. It’s not a problem. We all do, right? Even as consumers, it’s not a problem. And to come in with the low cost — like, if your value is that we’re lower cost, or whatever, that tends to be as soft as not standing up for the value that you’re actually producing. And if you have the proper go-to-market, and you have the proper product, and you have the proper positioning, then you can, basically, get the maximum dollars that customers are willing to pay. And everyone feels like it’s a very fair transaction, that the value being delivered to the customer is very reflective of the price that they pay.

      Jyoti: I think that the fair part is important. <Yeah.> Customers have to feel it’s fair, and you have to help them feel it’s fair, also, by making that case. Yes, the competitive dynamic will also <inaudible> on the price. If a competitor is selling for much cheaper, there may be some pressure on you to sell for that price as well, right? 

      In AppDynamics we had a bit of that challenge. One of our primary competitors was priced much lower than us. And they were designed more for SMB. Their product wasn’t as strong as ours for enterprise. And so, internally, sometimes people will come in — hey, our competitor is charging much lower, shouldn’t we decrease our price, also? And I’ll tell them, okay — do we really believe our product is superior? If our product is really superior, why would we not charge higher? So we’ve made a rule that we can always charge higher than them. We actually said that it would be always…

      Sonal: So you resisted the downward pricing pressure?

      Jyoti: Yes. Because, if our product is superior, we are — the customer is getting superior value, either we — either we are lying about it, or we are misguided about it, or whatever — that we believe it is, but it is not — or we are not articulating the superior value. That’s our problem. If our product has superior value, and we know how to articulate and make the case about it, why won’t we charge superior than them? And we always priced higher than our competitor because of that, and people are fine paying for it.

      Peter: And I think that, to further that point, the articulation of value often comes with having a sales organization. That’s what they do. And so when we often think about — hey, let’s don’t have a sales organization, so we can build more product —  often what gets missed in the whole product adoption cycle is the idea of selling value into the customer, where value is not necessarily felt through a self-service product, or whatever. You just can’t see the value or appreciate the value until an organization comes in to actually promote those pieces that may not be self-evident.

      Roles and responsibilities

      Satish: At the end of the day, it’s all about marketing — getting products to the market, right? And for that, you have the classical four pieces. So, product — it doesn’t go in isolation. Product, the pricing, promotion, and the place. At the end of the day, it’s the customer — with an intimate knowledge of that customer, and what exact pain process [they have] today, and how you want to change it in the future. Understanding that customer dynamic literally helps you define these four aspects. And that’s what a good founder, earlier on, or a good product manager — literally defines these metrics by understanding the customer.

      Sonal: So those four P’s.

      Satish: Yeah, that’s what a good product manager should be doing. I was running the product line — the newest business IQ product line, both product and business operations.

      Sonal: Is that an unusual model? Because you’re an engineer who’s doing product. Like, what is the ideal way to, essentially, architect the product management or product org functions in this framework? Are product managers salespeople, are they engineers?

      Jyoti: It depends on who your audiences are. To me, the product managers’ first job is to understand the customer and, you know — the classic definition, being the voice of the customer. So at AppDynamics, our product was technical. Our users were engineers, in many ways. So all our product managers had an engineering background. But if I had a consumer product — you know, I’m building out a fashion app — my product manager probably would be very good [at] understanding my consumers as, you know, someone who’s experienced in fashion. So for any business I’m doing, I would hire a product manager who can understand my end users very well.

      Sonal: So, kind of, matches the profile of your target customer?

      Jyoti: Exactly.

      Sonal: Did you guys have different product manager profiles, though, then — for the one-third of the organization that was doing the more evangelical startup within a startup next product line types, versus the ones that were doing the core? Because I would imagine those are two different sensibilities and they might or may or may not transfer.

      Jyoti: Yes, I would say the profile is a bit different. The product managers — once you have a v1 product, kind of, going from there, the profile could be a bit different. But at that point, you need multiple product managers. So you still want the product managers who could, you know, go and help create something — disruptively unique feature set, etc. But you also want, like, you know — product managers who are very good in understanding the, “How is it working out in the market,” and “What’s the adoption curve?” and “Is the pricing working?” So, you really — you know, the product management skill set also has different things to it, right?

      Sonal: What are the qualities to look for?

      Satish: We typically look for three aspects. During the initial phases — the empathy. To understand that customer, to define your product. And the second aspect is the business aspects of — okay, how is it going to work with the sale? So literally, the product managers, they travel with the salespeople and understand how do you position that value? Okay, now, how do I price it? And so on, so forth. And the third most important thing is execution. Once you define it, product doesn’t come out of thin air, right? You need to work with the engineers, literally attract your schedules, and really execute it and deliver it to the customers, right? So these are the three aspects — the empathy, and those business aspects, and, finally, the execution. So these are the three skill sets that I typically look for in a very strong product manager.

      Jyoti: There are very creative parts of product management. Then trying to come up with creative solutions is the second part, and then scaling the operation behind it — which is like a machine that can process the requirements from customers, from sales, figuring out the right pricing, packaging, all of that, right? So you want different skills.

      Sonal: It seems like a bit of a unicorn, to be honest, to have all three.

      Jyoti: And many times it’s not just one person, right? It’s — your product management then becomes a group at that time, and you want different people with different — like, that balances out the variety of skills.

      Sonal: Right. It’s just like a good team, you complement each other’s skills. And that’s the composition of an ideal team — why you have more than an individual contributor. Okay, so any parting takeaways given your — I’m sure you have a million takeaways, Jyoti, but any big message for our founders and other founders out there trying to do this, whether enterprise or not?

      Jyoti: It was a good discussion on the product-market-sales kind of fit, but my primary advice I will give to founders listening to this is — don’t overthink too far ahead, in many cases, as well. Like, the skills you need to master, $0 to $1 million of revenue, find the product-market fit, $1 million to $10 million, find the product-market-sales fit, iterate on it — let’s say $10 million to $75 million, scale the sales organization and go to your go-to-market. Then $75 million plus is when this — how do you build out product number two, and product number three, and product number four…

      Sonal: That’s a great framework.

      Jyoti: Anyone listening to this, I don’t want them to, like, you know — when they are in the $0 to 1 million stage, they’re trying to figure out how to do product number 2. There is no point spending time on that. So the skills that you have to learn and the organizationally — as an organization, and also as a founder — they change as you go. And my advice to people — focus on the thing that you need to learn the most to get to the next milestone and excel at it, then worry about the next one when you get there.

      Jyoti: That’s a great piece of parting advice. And it brings us full circle to where we started, in terms of how founders evolve as their companies do. And that’s a fabulous framework. Thank you for joining the “a16z Podcast,” Jyoti.

      Peter: Thank you all for this wonderful conversation.

      Jyoti: Thank you, Peter.

      Satish: Thanks, Jyoti.

      • Jyoti Bansal

      • Peter Levine is a general partner at a16z where he invests in enterprise companies. Prior, he was the SVP and GM of Citrix, where he joined via the acquisition of XenSource, where he served as CEO.

      • Satish Talluri is a deal partner at a16z where he focuses on enterprise companies. Prior to a16z, he worked at AppDynamics, Intel, and BCG, and founded Neptune.io.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      From Research to Startup, There and Back Again

      John Hennessy, Marc Andreessen, Martin Casado, and Sonal Chokshi

      The period from 2000-2016 was one of the best of times and worst of times for tech and the Valley (dotcom, financial crisis, Google IPO, Facebook founded, unprecedented growth, and so on), and John Hennessy — current chairman of Alphabet, also on the boards of Cisco and other organizations — was the president of Stanford University during that entire time. Given this vantage point, what are his views on Silicon Valley (will there ever be another one, and if so where?); the “Stanford model” (for transferring IP, and talent, into the world); and of course, on education (and especially access)?

      Hennessy also co-founded startups, including one based on pioneering microprocessor architecture used in 99% of devices today (for which he and his collaborator won the prestigious Turing Award)… so what did it take to go from research/idea to industry/implementation? General partners Marc Andreessen and Martin Casado, who also founded startups while inside universities (Netscape, Nicira) and led them to successful exits (IPO, acquisition by VMWare), also join this episode of the a16z podcast with Sonal Chokshi to share their perspectives.

      But beyond those instances, how has the overall relationship and “divide” between academia and industry shifted, especially as the tech industry itself has changed… and perhaps talent has, too? Finally, in his new book, Leading Matters, Hennessy shares some of the leadership principles he’s learned — and instilling through the Knight-Hennessy Scholars Program — offering nuanced takes on topics like humility (needs ambition), empathy (without contravening fairness and reason), and others. What does it take to build not just tech, but a successful organization?

      image credit: Jitze Couperus / Flickr

      Show Notes

      • The importance of RISC across technology, and how it began with a startup [0:00]
      • How the startup grew, and a discussion of changes in the startup space [12:29]
      • Discussion of the Stanford Model [17:51]
      • The importance of humility [24:20] and empathy [28:19] in leadership
      • Interdisciplinary studies [34:37] and the real-world applicability of AI/ML [40:32]
      • Academia-based research vs. corporate-based [42:42], and a discussion of talent in Silicon Valley [51:29]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I am Sonal. I’m here today with a16z general partners Marc Andreessen and Martin Casado. And we’re interviewing John Hennessy, who is the current Chairman of Alphabet and was President of Stanford University from 2000 to 2016, which also happened to be one of the most interesting times for tech and the Valley.

      So, in this episode we cover everything from the Silicon Valley and Stanford models, to if it’s possible to create other Silicon Valleys and, if so, where and how. And, of course, we also cover education, as well as the tech and economics of education, to what it takes to lead companies.

      John has a new book out, “Leading Matters,” on principles for leadership, and he also recently launched the Knight-Hennessy Scholars program for graduate students focusing on both knowledge and leadership.

      Finally, we discuss the evolving shift between academia and industry, including the role of universities, big company R&D, and the heyday of famous labs, and entrepreneurship then and now. Which, by the way, is why I asked Marc and Martin to join this episode, given their experiences going from university research to industry — Marc with Netscape, and Martin with Nicira, which came out of Stanford before being acquired by VMware.

      But first, we begin with John’s own history as a start-up founder, based on pioneering the microprocessor architecture used in 99% of devices today.

      Developing RISC and starting a company

      Sonal: So welcome, guys.

      Marc: Thanks.

      John: Thanks.

      Marc: I’d like to point out that Hennessy is also a Turing Award winner, which is unbelievably awesome.

      Sonal: That’s like the Nobel Prize of computing.

      Marc: It’s the Nobel Prize of computer science.

      Sonal: So, you and Dave Patterson won that?

      John: Yes, yes.

      Sonal: Why did you guys win it?

      John: Well, I think we won it because the work we did has reshaped the entire industry. Many times when you find a fundamental breakthrough, its importance may take a really long time to emerge, particularly in the hardware sector, it moves so much slower than software. And in this case, with the explosion of the mobile world and Internet of things, efficient processor architectures became really crucial. And that really changed the world. And that’s why our work has had such [a] great impact over time.

      Sonal: Well, actually, break down RISC for us. Like that’s “reduced…”

      John: “Instruction set computing.” The way to think about it is building a machine with a simpler vocabulary which can be executed more quickly. If you think about it in English language terms, imagine reading sentences that have giant $5-dollar words and are really hard to parse and understand, you’re constantly pulling out the dictionary. Now imagine a sentence that’s written in clear, precise English. Maybe it has a few more words, but you read it much faster. And we use that same key insight to try to build faster computers.

      Sonal: So, you reduce the instruction set in order for the computers to process information faster, and therefore operate faster.

      John: Cheaper, faster.

      Sonal: And why was that so cutting-edge at the time? I mean, weren’t the dominant players, like, IBM and DEC?

      John: IBM and DEC. And this is a time in the ’80s when, if you wanted to go talk to leaders in the computer industry and you were in Silicon Valley, the first thing you did was get on a plane and fly back east. It was a very different environment. They were building machines which were getting increasingly complicated rather than simpler. And they missed the whole importance of the microprocessor and VLSI and how we completely changed the industry.

      Marc: So, RISC was invented roughly when?

      John: Early 1980s.

      Marc: Early 1980s. And when do you think it really tipped to become as mainstream? It’s so mainstream today, RISC processors run almost everything.

      John: Yes, almost everything, except the desktop.

      Marc: Except the desktop and the server.

      John: And some of the servers.

      Marc: Some of the servers. But, like, every smartphone, every IoT device.

      John: Every smartphone, every IoT device.

      Marc: Every camera.

      John: You know, you probably own 100 of them that you don’t even know about.

      Marc: Right, right. So it’s, like, by far the dominant architecture today.

      John: Exactly.

      Marc: So how long did it take from inception to kind of when the market tipped to when we knew it was going to be absolutely dominant?

      John: You know, there was an early run at the mainstream market in the late 1980s and it almost flipped then. But what happened is, rather than the industry converging on one RISC architecture, they converged on three or four.

      Sonal: Oh, interesting.

      John: That gave Intel a real lead-up, because they didn’t have to beat any one, they had to kind of beat these little three or four.

      Marc: IBM, DEC, Silicon Graphics, Sun.

      John: Yeah, and they were all kind of beating up on each other, right? And so rather than getting behind one architecture, which would have made it much easier to build a software stack for it, that didn’t happen. So, there was a period where they were a lot faster, and then Intel really came back. And it probably took until the emergence of the cell phone.

      Sonal: What, that long?

      John: Yeah. Probably until mid ’90s. So, there was a period where it really wasn’t — it was working in the scientific computing space, but the scientific market is relatively small compared to the general-purpose market.

      Marc: But even so, right? The cell phone previous to the smartphone was not that  — yeah, it was a phone, it was great, it was a phone with a RISC chip, but it wasn’t a computer in the sense that we understand, right? So really the iPhone probably is the…

      John: Yeah, the iPhone was really the taking-off point. Some of the earlier Nokia phones began to use the technology. But then when the iPhone came along, boom.

      Marc: So, 1980-something, 1980, early ’80s, through to 2007 to really have…

      John: Yeah, to really have that big effect.

      Marc: Yeah. So, I just think it’s a great example of, like, these things are generational. Like, the really, really big things do take a very long time. But then when they tip, right? How many RISC chips do you think are globally today?

      John: Well, 99% of the market space. So, you know, it’s much larger than the number of — now, that’s counting processor chips, right?

      Marc: Right. But including embedded systems, 10 billion chips worldwide?

      John: Oh, more than that. Probably 50 billion.

      Martin: So, I worked for years under Pat Gelsinger at VMware, who was the GM of the 486 at Intel, and a longtime proponent of CISC. And he still maintains that CISC is the right architecture and, you know, dollar value, it’s still the predominant market or whatever. Are these different problem statements, or do you think it’s still just dying a slow death and we just haven’t got there yet?

      Marc: We should know, CISC stands for “complex instruction set,” so it’s the opposite of RISC and the classic Intel model.

      Martin: Right, exactly.

      John: Yeah, I think you have to separate out the technical argument from, “Does it have a large, established base and, hence, a large software stack?” I think on the latter point, Pat is exactly right, it has a large software base with large, established software. But in terms of things like energy efficiency, which now it becomes the primary concern. And as we get to the end of Moore’s law, and energy efficiency becomes more important, which you carry around a lot of devices in your pocket, they’re battery-powered. The fascinating thing people don’t realize is that after the cost of the physical servers themselves, the second biggest cost in a large data center is power.

      So, you care about energy efficiency even in these large data centers. And when it comes to that measure, the CISC architectures are far behind.

      Sonal: One of the things that surprised me is that the chips were used in early gaming systems, the PS4 and all these.

      John: Yes, that was one of the earliest breakthroughs for the RISC people in the embedded space, were games, high-end network switches, places where there was really — high-end color printers, where there was really a fair amount of performance demand, but also considerable sensitivity to price.

      Martin: Yeah.

      Sonal: So, why [were] games, like, the breakthrough then? The reason I think about this is because I think about what happened with GPUs and Nvidia, and how it then became the enabling for, like, artificial intelligence.

      John: Right.

      Sonal: More parallelized computing. So, I was just trying to figure out what the parallel was with the RISC story.

      John: So, one of the reasons was the RISC architectures — MIPS was the first architecture, along with the Alpha architecture, DEC, to get to a 64-bit implementation. And in the games, as in graphics, how quickly you can move data around makes a really big difference. And so, 64-bit architectures were much better at doing that. And that accelerated their — and first with, you know, Sony PlayStation being the first big breakthrough in terms of creating a much more realistic graphics framework for games.

      Sonal: By the way, is that why Nintendo is called Nintendo 64, because of the 64-bit?

      John: Yeah, Nintendo 64 is called from that.

      Sonal: Never connected those two dots. Back to Marc’s question though, what do you think made RISC tip? Yes, it took a long time, but how do you think — especially because in that time you founded a start-up, MIPS Technologies, to bring it to market. You could have just left it as a paper and expected the industry to adopt it.

      John: I was a bit of the reluctant entrepreneur. I mean, when we wrote our papers, we thought the evidence was so convincing that industry would just pick it up.

      Sonal: Yeah. I mean you said that about Nicira, I remember that.

      John: That’s what we thought. And, in fact, Digital Equipment Corporation actually had a research lab out here that took some of our ideas, some of the people who worked with us, and worked on the technology, but they couldn’t sell it back east, and that’s where the headquarters of the company was. You know, IBM canceled their project several times.

      So, eventually what happened was a famous early computer entrepreneur, Gordon Bell, who’d been one of the people that built Digital Equipment Corporation, came to me and said, “You know what? If you want to get this technology out, you’re going to have to go start a company.” And eventually he convinced me, although I have to say I was the technical entrepreneur that didn’t know the first thing about running a business, not the first thing.

      Sonal: We have so many founders who do that. What was, like, the biggest thing when you went to start a company that was like, “Holy crap, I don’t know what I’m doing”?

      John: I thought engineering should get roughly half the revenue. I didn’t realize how important salespeople really were. I thought, if you have a great product, people just buy it. So, there were a lot of things like that I didn’t realize.

      Sonal: Yeah.

      Marc: So not only did people not go ahead and build the products until you did, you had to start the company and build the products. Once you had built the products, they didn’t even just buy them?

      John: What we needed to do was find people who were — you know, companies are always a little reluctant to take a risk on a start-up, particularly with something like a new architecture, which really is a long commitment. So, what you had to do was find companies who felt like they needed a leg up over the other players in order to advance themselves. And that helped, we found a few players like that early on.

      Sonal: It’s kind of shocking that you founded your company in 1981, and we’re talking to founders in 2018, and it’s the exact same conversation.

      Marc: Yeah, you just described the exact same dynamic we see.

      John: But, you know, somebody said to me once, I mean, “What’s the difference between you and somebody else who’s read about technology?” I said, “Well, the people who’ve worked on it, they see the glass as half full, not half empty.” People said to us, “Well, that’s a nice academic experiment, but you’ll never be able to make a real product out of it, it will lose all its advantages when you try to engineer the rest.” Because we’d built a university prototype, it wasn’t a commercial product.

      Marc: There’s an old line, I forget who said it, but there’s an old line in the industry which is, “Everybody worries about protecting their idea. But if your idea is actually any good, you’re going to have to bludgeon people to adopt it.” Right?

      John: Exactly.

      Marc: This was a great example of that.

      Sonal: I think it’s interesting that you said that it was, like, a prototype, like research. Do you think that’s changed today where, because of all the systems that we have available to us — you know, AWS, all these different things where you can essentially prototype in the cloud — do you think that people now have more — when they are in a university of research lab, is their stuff more immediately and more easily transferable, because it’s more pre-industry scale or production-ready?

      John: Well, I think it’s probably a whole lot easier to transfer a software product than it is to transfer a hardware product. Software now, the students are incredible programmers, I mean graduate students, and you can really build something that’s pretty good shape. I mean when both Yahoo! and Google left the Stanford labs, they were pretty good pieces of software. They weren’t yet scaled up to deal with millions of users at once, but they were pretty impressive.

      Sonal: Yeah. Was that true for you guys, actually? I mean, when I think about Netscape, did you have to do a lot more work based on what you…

      Marc: Well, there were two things that happened. One is, when we were at Illinois, we started actually getting, like, people actually using our software, and then we ended up getting lots of customer support calls. And so we applied for an NSF grant to staff a customer support operation.

      Sonal: Oh, that’s hilarious.

      Marc: And the very nice people at the National Science Foundation explained to us that that was not actually the purpose of taxpayer-funded research. Which was a gift, in retrospect, and that catalyzed us in part to start a company. But then the other thing was we actually rewrote…

      John: You rewrote everything.

      Marc: We rewrote everything. And I actually think at Nicira, you guys did something very similar.

      Martin: Yeah, yeah, yeah.

      Marc: And so you do end up…

      John: You end up re-engineering.

      Marc: When you have paying customers, you do end up having to do a set of things that are not…

      Martin: Well, I always thought that was really interesting. And so my experience was very similar to yours, which I had these academic papers, the academic community liked it, industry hated it. And I found out it was actually much easier to sell somebody something than to give it away. And I don’t know what the psychology is about it.

      Sonal: That’s fascinating.

      Martin: This actually happened to me twice, where I’m like, “Oh, like, the paper is done, the research is done, I’m going to do the next thing.” Now I want someone to adopt it and I have the conversation, and then they won’t put the effort in or whatever. And in both cases I ended up just selling it to them, and in the cases of companies.

      And I think it does two things. One of the things it does is actually just qualifies. Because if you ask somebody for money, like, if they’re actually not interested, they’ll say “no.” And the second one, if you get a transaction to happen, you actually have some skin in the game, you actually have something behind it.

      And so, I actually tell this to a lot of academics coming out of industry now, I’m like, “Listen, like, it’s hard to give something away, it’s much, much easier to sell it, especially if you want to have impact afterwards.”

      Marc: What I propose, I propose the third rule from that, which is the more you charge, the more successful the implementation.

      Martin: 100%. And it will set the value.

      Marc: Right. Because the more painful it’s going to be for them to write it off.

      Martin: Totally.

      Marc: And so they have to commit.

      Sonal: Right. That’s your two-word mantra, is, like, “Raise prices.”

      Marc: “Raise prices.” It’s another great example.

      Startup challenges

      Sonal: Well, you know, Nicira — and before you guys were acquired by VMware, I remember you wrote about how you guys actually had some early adopters, but then you had, like, sort of a hump. And you talked about, too, how you had an initial fast — and then you kind of stall. And so one question I have is, like, when you get to that moment, coming out of academia and then into industry, what sort of tipped you over to sticking it out, and then figuring out how to get over that hump?

      John: Well, we had a situation where we had probably expanded a little bit fast and the first CEO — remember, this is a bunch of three technical founders who didn’t know anything about really running a company. He had expanded too fast on the evidence of the first customer and, you know, we had too many people. And we were about to run out of cash. So, we had to kind of do a reset on that. We had to go through a layoff, which was a really tough situation. 120 people, you got to lay off 40 of them, you know everybody. And then the CEO asked me to get up at the Friday TGIF and give the rally call for the company — how we were still going to be a great company and this was a small hiccup on it. But I had to learn from that process and re-energize the company.

      Sonal: I mean, your whole book is about leadership lessons. What was, like, the biggest leadership lesson in that moment?

      John: Well, for me it was if you have a crisis and you’ve got to take a tough step, do it quickly, get it over with, and move through. Reset the clock so you can then charge ahead. And that turned out, when the financial crisis hit, you know, Stanford lost billions of dollars of its endowment, about 28% of the endowment vaporized in a six-month period. So, there was no way we could continue to spend money the way we were, we were going to have to go through that process again. I realized, you know, that’s going to lead to 5 or 10 years’ worth of small budget cuts that are going to not be very efficient, and we’re going to not be able to do anything new. So, we sat down and said, “We need to do this quick.”

      Sonal: So instead of death by 1,000 cuts, you’re going to do, like, one hard stab.

      John: Yeah. We did it quick. “We’ll be generous, we’ll be humane, we’ll give nice severance packages, and then we’ll restart and begin to rebuild the financial core of the university.” We had one year that was sort of a down year, and then we’re back.

      Sonal: Yeah, that’s great. You started a company in the ’80s. And you started a couple of companies, in fact. And you IPO’ed only five years, I think, after starting your company. And today a lot of companies don’t IPO so quickly, so that’s one big trend shift. What are some other shifts that you’ve seen, especially since you counsel and meet a lot of entrepreneurs, between then and now?

      John: I think probably one of the biggest shifts — the space of start-ups has changed dramatically. You know, when we were starting, our goal was to build a product that was more efficient, that solved some particular problem. Now, with so many software companies, the whole big question is, you know, “Will the dogs eat the dog food?” I mean, is it really going to get traction, is it going to go viral? I think that’s a very hard thing to predict ahead of time. I mean, look, I was sitting at Google when Facebook came along. Nobody foresaw how big social media — I mean, some did. Mark did, clearly. A few other people. But most of us didn’t see how big it was going to be. And that happens all the time.

      Martin: Yeah, it’s interesting. Even enterprise companies now are having this type of characteristic. So, it used to be the case, you’re like, oh, a consumer company is kind of a popularity contest. You’ll have three companies that all look the same. One will get adopted, two won’t. But the enterprise was kind of core tech, and then you could actually talk to the buyer, and then you could predict somewhat whether it’s going to do well or not. Or at least whether a category is going to do well or not. But what’s happening now is, especially because developers are so influential in the enterprise, and developers are also kind of fickle and, you know, they have their own philosophies and so forth, whether or not a company is going to do well is somewhat independent of technology often, and somewhat independent of the approach they take. And it’s more like, you know, “Do they become the popular one that they use?” So, I think this is something we see across the industry.

      Sonal: Yeah. People in the enterprise, it’s not just developers. You guys talk a lot about, like, departmental-level buying even across…

      Martin: Yeah, yeah. Vertical SaaS, that’s right. Yeah, yeah.

      Sonal: Yeah, exactly, it’s coming from the bottom up.

      John: But I think even in complex organizations, universities like to have a very slow, deliberative process. But in a complex organization, all decisions are gray when they get to the top. And so you’ve got to get comfortable making decisions, making calls in that situation. And I learned that in the start-up environment. And I wouldn’t have learned — it would have taken a long time to learn in university.

      Sonal: Right. Well, what do you think about — we have this view that professors that are part-time co-founders, I mean — we don’t believe that when a professor is listed as a cofounder in a company, that if they’re a part-time — that they’re actually fully committed. We need to see more skin in the game.

      Martin: Having lived through this.

      Sonal: Oh, did they tell you the same thing, were you trying to do this part-time thing?

      Martin: No, no. I had two part-time professors and I was full-time.

      Marc: Yeah, you were full-time.

      Martin: Yeah, yeah.

      John: Yeah, you had two part-time professors, right?

      Martin: Yeah, I had two part-time professors. I mean here’s the reality — start-ups require a tremendous amount of work and effort and time, and you make real commitments to customers and teams and investors. And early on, while you may have a great idea, the investment is in you. And so there’s really a mismatch in expectations between someone giving you money, a team coming to join you, if you’re not going to be there long-term.

      And so, we like to know, if we’re investing in someone — whether they come from academia or not — that they’re going to stay with the company for the duration of, kind of, the team and the investment. Now that doesn’t mean that a part-time professor doesn’t come in and help out, right? I had two, and they helped out a tremendous amount. But what we like to see is someone that is fully committed.

      The Stanford Model

      Sonal: What advice would you give to universities who are trying to do something like the “Stanford model”? Which, I don’t even know if we defined what the “Stanford model” is, but it’s pretty cutting-edge — and we take it for granted in the Valley that Stanford and Berkeley, for that matter, will give away more IP than they hold onto. And I used to see, when we were at Xerox PARC, a lot of university tech transfer offices. And it’s so extractive.

      John: Right.

      Sonal: And kind of nightmarish, in fact.

      John: Right. Marc has the great experience at doing this, but my view of — people think of their technology licensing office as extracting blood, as opposed to being partners with their entrepreneurs. And the purpose of technology licensing, from a federal government’s viewpoint, is the university should get their technology out there. If they focused more on that, that would be great.

      And be more flexible with respect to faculty. My experience is, the faculty members I know at Stanford that have gone out and started companies are better researchers, they’re better teachers. They’re all around better, because they have a wider range of experience. And most of the students we educate, they’re not going to become future academics, they’re going to go work in industry. So, a faculty member that has experience from that is actually a better teacher.

      Marc: So, let me play devil’s advocate. Which is, okay, that’s all fine and good for you to say, but we only have so many professors. If they go leave and start companies, like, they may or may not come back, they’re distracted, they’re not teaching, they’re not doing research. Then aren’t we depleting the core mission of the university of doing research and education by enabling that?

      John: It’s a good question. I think we’re in a tricky position right now, especially around the machine learning/AI area, where there are lots of faculty who are leaving. And that will hurt the industry in the long-term, because that means we’re eating the seed corn. I’m a great fan of faculty members who go out, commit themselves to a company for some period of time, but say clearly that their long-term goal is to go back to the university. That works well. I think if all the faculty leave, then we will have a problem long-term.

      Marc: But there’s also some, presumably, benefit to being the place where people feel like they have a lot of flexibility, the place that encourages creativity, the place that encourages ventures, that presumably will play a role in attracting.

      John: Right. So, you’re a young person, you’ve got multiple faculty offers. You might be interested someday in taking your technology out. Where’s the place to come? Well, it’s pretty obvious where the place to come is, and that’s a big benefit to the university in terms of recruiting people.

      Marc: And so, we all the time get the delegations from, you know, various countries, various cities in the U.S., various countries outside the U.S., and sort of the question is, you know, “How do we create Silicon Valley of X?” It could be “Silicon Valley of Chicago” or it could be “Silicon Valley of France.”

      John: Kazakhstan.

      Marc: Or Kazakhstan or, right, anywhere, anywhere. And I’m sure they come and see you, as well. And so what is your answer to that question?

      John: First of all, build some great universities, because they are a center of innovation, and many of the ideas which build not just a single niche company, but help transform an entire industry and create an entire industry coming out of universities. Build the rest of the ecosystem out. I mean, the fact that venture was out here and people were comfortable with it, the fact that you had legal firms who knew how to work with start-ups and make that work. But risk tolerance is a big part of it. You can fail in the Valley, provided you had a reasonable strategy and a reasonable set of goals, and reboot — and it works okay. That’s not true in many parts of the world.

      Marc: So maybe let me polarize the question a step further. So, the cynical view would be you can’t. You can’t create Silicon Valley anywhere else because there’s only a couple areas of technology where it’s even feasible to create a Silicon Valley, and Silicon Valley already has information technology. And then further, the things that you just described, like, they’re just too difficult to do. It’s very hard to create a new research university from scratch, it’s very hard to change the culture of the country that you’re in. That’s why there’s only going to be a handful of these places.

      The optimistic view would be, “No, no, no. All these ideas are now spreading, the world is globalizing, technology is globalizing, the knowledge of how to do all these things is globalizing.” And then there’s many new areas of technology that are becoming, kind of, more amenable to this kind of flexible innovation, and many countries that, you know, want lots of entrepreneurship, and many kids worldwide who are growing up watching YouTube videos of, you know, Stanford classes on how to build a start-up, and then, you know, getting out their compiler and getting to work on writing code and starting their companies. And so, in that positive vision of the world, there’s, you know, 80 or 100 Silicon Valleys in 10 or 20 years. Where do you come out on that?

      John: I don’t know that there are 80 or 100. So, it is going to happen in China, I have no doubt about it. The government is pouring enormous amounts of money into building their top half dozen research universities. The people are very entrepreneurial, there’s a lot of risk capital available. There may be some issues around liquidity and exits that are a little difficult, but they’ll work that out over time.

      It surprised me that nobody in the U.S. has built a real competitor. In fact, just the opposite has happened over time. If you were to ask me 15, 20 years ago, “Will there be another Silicon Valley in the U.S.?,” I would have said, “Yes, for sure.” In fact, just the opposite has happened — the Valley’s lead has gotten bigger.

      Now, we may be the victims of our own success, given land and traffic and cost of housing. We may be laying the foundation for some other Silicon Valley area, but it’s got to be a place where people want to live. And that helped bootstrap it. And so, we should be looking and thinking, “Where is that going to happen next, where is that a kind of opportunity?”

      Marc: Do you think we’re at risk of strangling our own success by all of the fundamental issues around housing, transportation?

      John: I think we are.

      Marc: Taxes.

      John: I think we are.

      Marc: A state government that seems to hate us. A city government in San Francisco that seems to hate us…

      John: Yeah, I think we are. Or hates us and loves us at the same time, right? You know, our cities and the state have such dramatic issues. And yet, you pull out the high-tech sector, I mean the state and the city of San Francisco will collapse.

      So, we’ve got to think about it. And it really — you know, the younger generation moves to this area, but without that kind of suburban dream of, “Oh, I need the large house with the lawn.” I mean, they’d rather have something maybe a little smaller, not have the big yard, to have some nice parks, have some open space, and, by the way, be able to walk to three restaurants and a movie theater. And that’s a different view than the Valley grew up doing. Then you’ve got to figure out how to make the transportation network. It may be that rather than rely on government, we’ve got to get the companies to play a much bigger, forceful lead in pushing governments to do the right thing.

      Sonal: I mean, one could argue that’s what’s already happened with the shuttle system.

      John: Yeah, the shuttle system is that.

      Sonal: As, sort of, this private tunnel.

      John: It’s a patch.

      Sonal: Right, it’s like a patch, exactly, into, you know, this public infrastructure. The newest trend that I’ve seen, because I am friends with a lot of 20-year-olds — they are doing a lot of cohousing arrangements, where they’re all renting big houses with like 20, 15, 10, 8 people. And our friends would never have thought of doing that when I was in grad school and undergrad. It would have been, like, two roommates at most.

      John: Yeah. I think when I see a lot of the start-ups coming, I mean that’s what they’re doing. They go rent a house and squeeze more people into it than you ever thought were possible, right?

      Sonal: Right.

      John: But it doesn’t matter because they’re working 60, 70, 80 hours a week, so…

      Humility and empathy in leadership

      Sonal: One question on the note that Marc was asking about the next Silicon Valley. So, the network effect of it becoming more valuable the more people that are there — the other part of the ecosystem is obviously people who are, you know, like yourselves, ex-founders, ex-salespeople, ex-marketing heads, etc. — who can then help these companies as they grow and get to the next level. That’s the biggest argument I’ve heard for why there might not ever be another Silicon Valley.

      John: That’s a great argument. I remember a start-up founded at Marc’s alma mater, at University of Illinois. And — great group of people, they could hire great young engineers, because it’s one of the best engineering schools in the country, but they couldn’t get the kind of middle and upper-level management there.

      Sonal: Right, exactly.

      John: And so they ended up moving the company to the Valley, because there was lots of depth there.

      If you look over history, I mean, Hewlett-Packard was there, then talent from Hewlett-Packard helped build Sun, talent from Intel helped build the first generation of fabless semiconductor companies, and that spread out over time. And that’s one of the great things that happens in the Valley.

      Sonal: I agree. And I know this sounds so hokey, but I’m going to say it because I don’t think people really appreciate how unique it is. The generosity of mentorship. And, you know, a big theme of your book is about mentoring and molding the next generation of leaders, so let’s transition to talking about what some of [those] mentoring and molding principles are.

      So, each chapter is devoted to a specific principle — humility, empathy, you know, honesty, transparency. There’s different levels of that. But they’re things that everyone says about leadership. So, I’m going to challenge you to convince me — what is the nuanced take on why humility matters? And by the way, on that one especially, I don’t know of that many humble leaders, quite frankly, that are really successful.

      John: I think you can succeed while being humble if you’re also ambitious at the same time. Classical person who’s humble and ambitious is Abraham Lincoln. He’s just got to maneuver things over an extended period of time, he has to go to war, but he was a very humble person. I mean, and I think that combination — what humility does for you, is it removes the barrier to asking for help, to admitting that you’ve made a mistake. Which, for many people, that’s a fundamental thing. Look how many of our leaders won’t admit that they made a mistake, right? And won’t ask for the advice of others.

      Marc: I think the challenge that leaders confront on that is, “If I show weakness, my people will start to lose faith in me.” And so what do you advise a leader who’s worried about that?

      John: I think there’s a difference between being humble and being indecisive.

      Marc: Okay.

      John: And I think it’s a question of making that decision. You know, when Abraham Lincoln finally drafted the Emancipation Proclamation, the majority of his cabinet didn’t want him to publish it, didn’t want him to release it. And yet he knew that that was the moment — that that was the time he had to do it, that he had to make that decision and move forward. And I think that kind of decisiveness is crucial.

      So, you’ve got to take responsibility for making the decision and moving forward, but that doesn’t mean you shouldn’t gather all the input and be open. If you’re humble, then your staff, your team can come up and say, “You know what, Hennessy? That’s a really stupid idea. And if you do that, it’s going to come out bad.” Then you say, “Okay, well, you know, you’re probably right, I need to rethink this.” That’s fine.

      Sonal: It’s kind of like our “strong opinions, weakly held.” Which feels like a very a16z value — it really seems to define the place. I love this phrase that you use in your book, “It’s not enough to understand how many people are depending on you, it’s just as important to realize how you are depending on them.” And I thought that was a very neat thing to think about — mentally inverting the org chart.

      John: Yeah. I like to think of my org chart upside down. I’m the person supporting the rest of that team and serving them.

      Sonal: I always think of how this plays out when it comes to things like equity, though, because you have to share the success. But, you know, quite frankly, some people do more, some people do less, some people are less fungible, others are more, and you have to take that into account. And I think that’s sort of an interesting calculus that people tend to sort of balance.

      John: Well, you have to think about the value of the individuals. Everybody’s work has value, but obviously some of it is more crucial to the success of the organization than other work. So, everybody should be rewarded, but that doesn’t mean all the rewards should be equal.

      Sonal: Let’s talk about empathy. Because you’re one of the pioneers, in your tenure as president, of the largest increase in financial aid ever, which allows more lower-income families to experience Stanford. And this is incredible. But you talk about how it was hard for you to actually make this happen, because empathy needs to be balanced with fairness. And that really resonated. So, tell us about how you sort of navigated that thorny issue.

      John: So, we decided that one of the challenges that people who came from disadvantaged backgrounds faced is just getting through the whole process of applying to a highly selective school. You know the federal financial aid form is 23 pages long? Often you get people — they may not even speak English because they’re an immigrant family. And so that’s a major barrier. We decided we needed a very simple message, right? Your family makes less than $100,000 a year, your tuition at Stanford is $0. The next thing that happened, though, was somebody came in and said, “Well, I make $110,000 a year and my tuition is $30,000 a year. This doesn’t make any sense.”

      So, we concluded you had to balance this with fairness. You had to ask the students to have some skin in the game.

      Sonal: Right.

      John: So, we said, “Even though your tuition is $0, you have to work for the university 10 hours a week during the year, and 20 hours a week during the summer, and contribute that to your education.” And then everybody said, “Well, that’s fair, that’s reasonable.” So, balancing that was really key.

      Marc: So, can I ask you the obvious follow-up question?

      John: Yeah, sure. Yeah.

      Marc: So, how many 18-year-olds a year — how many kids come of age to be 18 in the world each year right now?

      John: Oh, a gigantic number. I don’t know, Marc.

      Marc: About 100 — I don’t know.

      John: Yeah, yeah. A very large number.

      Marc: 100 million, some large number like that.

      John: Yeah.

      Marc: How many undergraduate freshman slots does Stanford have each year?

      John: About 1,750 this year.

      Marc: Yeah. And how many total university slots are there globally in Stanford-scale institutions, or Stanford-quality institutions, for the freshman class?

      John: Well, let’s say — I mean, then you’d have to put all the elite publics in. I mean I’d say, probably there are maybe 200,000 slots in the entire United States.

      Marc: So, take 100 million 18-year-olds to 200,000 slots. You know, the obvious question, right? Which is, like, it’s fantastic, obviously, what Stanford is doing for the kids who then end up in Stanford, but most kids don’t, most kids don’t end up in anything resembling a Stanford-quality education.

      John: I came to the view that the university had a moral imperative to increase the size of the student body. Now, there’s a limit [to] how far you can increase it before you change the quality of the experience, right? We house all our students on campus, things like that. But we could certainly do more. And the provost and I made an argument.

      So, in the end what happened — the financial crisis came along, we had to put that on the back burner. But then it came back later, and we’ve engaged in the gigantic expansion of undergraduate housing so we can house students on campus.

      Marc: This does sound a little bit like the Director of the Globe Theatre in, you know, 1550 or whatever, kind of, saying, “More people should get exposed to Shakespeare’s plays. And so therefore we should build a balcony, right? And we should, you know, double the number of people who can come to London and see the play.” But, like, most people in the world are never going to be able to get to London and see the play. Like at some point isn’t the right answer to invent television?

      John: No, the right answer is to change the way we educate people. I mean, I think if you were to make an accusation against higher education, it’s that they haven’t really done very much to bend the cost curve. And part of this is understanding what it means to bend the cost curve. Think about Vivaldi writing “Four Seasons” and having four musicians play the “Four Seasons,” right? It takes 23 minutes. It took 23 minutes in, whatever it was — 1790s, it takes 23 minutes today. What’s the big difference? Those musicians get paid a lot more today than they got paid then. So, actually, there has been no productivity gain in the presentation of the “Four Seasons” piece, right? And universities are somewhat in that, it’s still a craft to some extent.

      Now, that has to change. That has to change. We’ve got to figure out how to leverage technology in an appropriate fashion to get the cost of education down. Otherwise, it’s simply going to become more and more expensive for American families, we’re going to load up with student debts going through the roof. And part of the reason it’s going through the roof is families are less able to save than they used to be, and so we see student debt going up.

      Marc: The one form of debt that is not discharged through bankruptcy?

      John: Yeah, correct. But it’s also — look at the default rates. Now, part of this is the for-profit industry, unfortunately, in the higher education space doesn’t deliver a lot of value. So you end up with lots of students who are not able to use their education to get ahead. We’ve got to figure out how to deliver a high-quality education. Not decrease the quality in order to just get the cost down, but hold the quality up while reducing the cost. And the only way I know how to do that is by using technology.

      Marc: Have you read Bryan Caplan’s book, “The Case Against Education?”

      John: No, I haven’t read it.

      Marc: It’s probably not a common book on the Stanford campus. Although he is a tenured professor of economics, and so he is an instance of what he is talking about. And so, I’ll just focus on one aspect of the book that he talks about. The sheepskin effect, if I recall correctly, is basically if you take somebody — if you take an undergrad who’s completed seven out of eight of their semesters, right? So, they’re three and a half years into their program and they drop out. You might think that they would get seven-eighths of the income in their first job as somebody who does all four years, and it turns out that’s not the case at all.

      John: Right.

      Marc: Which then, basically, means that the value of that four-year education program is primarily in the signal of the diploma, as compared to the actual education. I think statistically, I think, this is in the numbers. So anyway, you might interpret that in different ways. I’d be curious how you would interpret that.

      John: I think there’s some truth to this observation. And I think one way of interpreting it is that the drive and the determination to finish that degree is actually the key signal that employers are looking for, not just what courses you took.

      Now, I should say, post-bachelor’s degree, this is changing dramatically. But if you think about other kinds of post-bachelor degrees, we’re moving very quickly towards a certification type model, where you take a course, or a sequence of courses, right? So, you go and take the sequence of courses on cryptography and blockchain, and you become an expert on that. And by demonstrating that you’ve mastered three, four, five courses, then that all of a sudden becomes the key to getting a new job opportunity. I think we’re going to see more and more of that as we go along.

      Marc: So that’s, like, an alternative to a master’s degree?

      John: Yeah, it’s an alternative to a master’s degree, you actually have to demonstrate mastery of the material. I think that’s the key thing, and that’s what an employer wants to know, right?

      Sonal: It’s like Udacity with the Nanodegrees to some extent, too.

      John: Yeah, it is like that.

      Interdisciplinary studies and degrees

      Sonal: Actually, on this very note, like, I would love your take on the interdisciplinary side of things. Because to me, the one unique thing that universities can do that a lot of these other institutions cannot do is break down barriers between disciplines. And you guys have tried experiments, or legitimate degrees, like symbolic systems, etc., that cross across, you know, multiple disciplines. But I’ve yet to see examples of true successes in multidisciplinary degrees or entities. Like, maybe Xerox PARC would be the best example, but I really can’t think of any others.

      John: Happens a lot more at the graduate level and the research level. Partly because I don’t believe that multidisciplinary or interdisciplinary things are a substitute for some deep domain knowledge. I’m a firm believer that you start with deep domain knowledge, and then you build on top of that.

      You know, one of the challenges with these small courses that certify you in an area — those work well for a professional. They’ve already got an undergraduate degree, there’s a clear connection between the value of the education program and how they’ll be rewarded.

      Take an undergraduate coming in without some of the advantages that you’d have if you want to an elite high school. They’re not going to thrive very well in that kind of online setting, where they don’t see how that directly translates to getting a job at Facebook, for example, all right? They’ve got a long way to go before they’re there. So, they need a rather different educational system than somebody who’s already got their degree. They see, “If I take this course, I’ll get this new opportunity.”

      Martin: I also think computer science is a little bit unique in this, in that, you know, so we call it a science, but, I mean, ultimately it’s an engineering discipline. And while there is, like, pure computer science, almost all of it is applied. And so, when I did my Ph.D at Stanford, we had people that would work in graphics, and they worked very, very closely with, you know, computational physics, for example, solving very real problems. Same thing with biology, right? One of my best friends, I mean, he did some really core work in DNA sequencing. And if you squinted at him one way, he looked like a biologist. If you squinted another way, he looked like a computer scientist.

      The thing that I love about computer science, and I’ve always loved, is if we wrote a program that solved grand unified field theory, physics would go away as a discipline, and we’d be like, “Okay, that was more application. Let’s go on to biology,” right? So, in some ways it doesn’t exist without, like, the other disciplines, in another way it really is kind of this meta-discipline. And so I do think it’s pretty unique in that way.

      John: It is unique and it is this meta-discipline, I mean, I think. And it’s become the new meta-discipline that everybody needs to learn.

      Martin: Exactly.

      John: Because algorithmic thinking is such a fundamental thing about how the world operates these days.

      Sonal: Right. Like math, reading.

      John: Like math, right? It’s just like that.

      Sonal: You know, computational literacy should be just one other form of that. I was thinking, there was this debate with Vitalik Buterin — who’s, like, the inventor of Ethereum — and this professor, who’s a former editee of mine. And the debate they were having was whether there should be a dedicated degree for blockchain. So, the professor was saying, “We don’t need this, you should have fundamental basic science, and that’s good enough.” And Vitalik’s point was, “Well, actually, this is a really interdisciplinary, multidisciplinary, unique case where you’re layering economics and computer science and lots of other — finance and lots of other things, in a very intersected way.”

      So, I thought that was fascinating, that there was a sort of tug of war. And this, to me, is the wave of the future. Like, I could even see the blockchain as a laboratory for people learning on their own in the future, especially if you think about what Marc mentioned earlier, about all these kids coming online around the world who don’t have access to these universities locally and are learning from YouTube. I could see programmers in my parents’ village in India becoming people who become such experts in this world. I mean, you’ve been the president of a university for 16 years that I greatly respect, but I wonder if it means that maybe the university model might have to really evolve in a different direction.

      John: Well, I think there’s about to be a great test of this, because [of] the wide applicability of machine learning to all kinds of problems, all kinds of problems. I mean, you know, you should see breakthroughs in biology, in chemistry, in astrophysics, coming out of various forms of machine learning. So, all of a sudden it becomes this tool that is applicable to a whole range of things and is changing those fields. What do the scientists, the people who think of themselves as astrophysicists or as organic chemists — how much do they need to understand, how do they deploy this technology?

      And this is a big gap right now, because the senior people in the field — it’s highly unlikely that most of them are going to take a year or two out and go back and learn a bunch of things about computer science and statistics and machine learning ideas. We’re really going to have to build a new breed of people who, kind of, fill up this interstitial space and become the key innovators in the disciplines.

      Sonal: Well, I would argue that it needs to be more applied. We have an executive briefing center with a lot of big companies coming in, and the number one challenge they have when it comes to ML and AI is production-ready, industry-applicable machine learning. It’s actually, like, what’s happening in academia is not at all connected to what they need to actually do.

      Martin: Yeah, it’s not only that. Which is as you move to AI and ML, more and more of the value is the data.

      Marc: Absolutely.

      Martin: And more and more it’s, you know, almost serendipitous understanding of the data prior to manipulating it, right? It’s almost impossible to remove the context of the domain understanding from data. From programs, maybe. From data, almost certainly not. Which is why we’re seeing such, kind of, a confluence of CS, statistic and data understanding, and domain expertise.

      Sonal: Right. It also goes to your views about the end of theory. You should share that.

      Martin: Or not.

      John: So, you’ve got to look at that. You’ve also got to look at how and who establishes ground truth in these. I may have an AI program that can recognize some medical condition, but who decides whether or not it’s right on the basis of that? ML is the ultimate “garbage in, garbage out” technology. Because if the data isn’t good and properly validated and the learning process isn’t — you’re going to get assumptions and outputs that are ridiculous.

      Martin: So, this is something that we have to deal with a lot, you know, in venture capital, which is a number of constituencies and entrepreneurs actually view AI or ML as almost, like, the end of theory. So, it’s almost like, I don’t have to know what I’m doing — the AI and ML will figure it out for me. So, like, they’ll come in and they’ll say, “Listen, there’s all of this data in enterprise X or whatever, we’re going to apply AI and ML, and then the net result is going to be value.” And, like, “Well, what’s that value?” “I don’t know, the AI and ML is going to tell you and it’s going to be valuable because we’re going to apply this.”

      And so, like, it’s a very important toolset, but I think you have to understand the domain. To your point, “garbage in, garbage out.” You have to have some way of getting the expertise or whatever in the prior to get the answer. It’s not like this has become the end of theory, and we don’t have to know what we’re doing anymore and we’re going to get valuable results.

      John: And the space where that works, sort of unsupervised learning, is such a small part of the giant ML space. It’s relatively small. And most of its interesting applications are in the natural science world, not in real-world applications.

      Martin: Where there is actually a truth and way to test the truth, right?

      John: Exactly.

      Martin: And so for me the most difficult thing about moving from academia to industry was that in academia, you look at a problem domain and, kind of, your job is to think very, very clearly and, like, pull out, like, these kind of, you know, global truths, and they have to be very elegant. And very rarely do you write a paper where you’re like, “Here’s this problem domain and here’s, like, my litany of 50 fixes. And read through every one of my heuristics and, oh, look how elegant it is.” Right? It’s almost the exact opposite. What you learn about starting a company is, it’s actually the opposite — which is, almost every solution is dealing with a heavy tail of complexity, and it’s a bunch of patches and the real world and everything else.

      And so mentally you’ve got to go from, “I’m going to look at a problem space and extract elegance,” to, you know, “I’m going to deal with all of this complexity and master it.” But where I did find this energy very useful is, a lot of leadership is thinking simply. And so if you start a company and you can extract that elegance, you can use that to really lead a company, and you can convince a customer, and you can talk to an investor — because you’ve really distilled what’s important about it. But you can’t let that constrain you, because ultimately you have to build something that solves a real problem, and the universe is a messy, messy place.

      And so, if you can get beyond that kind of ability to have everything be incredibly elegant, I think you can have both the leadership and kind of, like, the actual complexity.

      Sonal: That’s fascinating.

      Academia vs. the corporate world

      John: Yeah, no, I think you’re absolutely right. I think in the academic world, we like things that really look elegant. And we often actually delay publishing a paper or getting a result out there until we get it all gelled just right, right? That doesn’t work in a start-up company.

      I think the one thing that is common is — focus really does help in both cases, right? I mean you’re relentless in a start-up company, you’ve got to focus, you’ve got to drive, you’ve got to decide what’s peripheral, and [that] you’re not going to do now. And the same thing is true in academia. If you want to do really great work, you need to focus, you need to kind of — somebody once told me, they gave me some good advice. They said, “You know, you ought to be working on three or four things, but you ought to have one or two of them that are really important, where you’re really putting your energy. And these others are your backup in case those really great things don’t work, and you don’t get tenure for those.” And that was good advice about how to think about a research career, but it doesn’t work in a company. You’ve got to get rid of those things that are not the home runs.

      Sonal: When I think of examples like Xerox PARC, which honestly, despite the mythology, they actually did put a lot of repeat successes out into the world. It wasn’t that they had, like, a carte blanche to just invent whatever they wanted. They had a very specific mission, and they invented towards that mission. When you talk about the differences between academia and industry, academia is about ideas and industry is about implementation. And you believe that there’s an interface that VCs and others carry across those two. Do you think, though, that that’s sort of a false divide in some ways? So, it was actually not just ideas versus implementation, it was ideas in practice, in industry settings — because it was for a corporate research lab. So, I just wonder how you’re thinking about this — was then and now, and how it’s evolved.

      John: So, I think there was a time when IBM Research, Xerox PARC, and Bell Labs were the great giants.

      Sonal: Yeah.

      John: What they had — they were not devoid of application and things. I mean the work on the transistor was really begun to solve a fundamental problem that a telephone switch built out of tubes. What they did have was, they had the advantage of a long investment horizon. It’s harder to find that in industry nowadays. It’s harder to find that patience. Partly because of the observation that, if you discover something really big, lots of people have to eventually benefit from it, right? Bell Labs and AT&T were not the major beneficiaries of the discovery of the transistor. Xerox was not the major beneficiary of the discovery of modern personal computing, right? That’s why universities are the ideal place to do this kind of work, because society benefits. Universities do technology transfer in a very natural way. It’s called graduation.

      Martin: Marc and I are both dying to jump in. I think historically that’s certainly been the case. One could make an argument that this is shifting, and some of the most fundamental research contributions are actually happening in industry today. And not only that, that — you know, the academic system has actually moved towards short-termism, especially in incremental publishing. Like, I even feel like I’ve seen that dynamic shift in the last 15 years in just my, kind of, professional career. Where I would say Google and Microsoft are doing some of the more, you know, innovative fundamental contributions. And then I still sit on program committees — it’s interesting, they publish a paper, I’m in the PC committee, and then all of the professors are basically trying to do incremental work on top of Google’s work, right? So, are we seeing, like, an imbalance lately, or is this a momentary thing?

      John: No, I think you’re right, I think there is a bit of a shift occurring here. It’s driven by not only the amount of resources that are available at Google, Facebook, Microsoft. It’s driven by data, and it’s driven by computational resources that are available in those companies that are much larger than is available to a typical university setting. So, I think we’re seeing a growth of, kind of, [a] new research environment in industry that’s quite a bit different than the old environment, and may be a harbinger of how things get invented in the future.

      Marc: I’m kind of the skunk on this topic. So, I think the reason — the skunk at the garden party. So, I think the reason — I mean they did great work, Xerox PARC, Bell Labs, IBM Research. But here’s the thing, like, it’s always those three examples. They’re basically like they were rounding errors on everything. Like there weren’t 10, there weren’t 20, there weren’t 100, there were 3 or 4. And there were two preconditions for them. One is they all were offshoots of monopolies.

      John: They were all offshoots of monopolies, you’re exactly right.

      Marc: So your point on long-term thinking, the reason they had long-term thinking is because monopolies…

      John: They could afford it.

      Marc: By definition, all monopolies have this long-term thinking. Right?

      John: They all were offshoots of monopolies, that’s an important insight.

      Sonal: I never thought about that.

      Marc: And arguably from a corporate, like, investment of capital standpoint, they were worth it just for the marketing value. Right? Of being able to demonstrate that they weren’t just, you know, sitting on their rear ends in the corporate office. And then, the other precondition was they were all pre-1975, 1980 — they were all pre-venture capture capital.

      John: Yeah, yeah, yeah.

      Marc: Right? And so when the monopolies cracked, and then venture capital pulled the talent out, like, that was basically it. And the downside case would be, that removed this kind of long-term commercial research. But the upside case would be, that led to what I would argue as just an explosion of R&D at far greater scale, right? Across the corporate landscape than ever existed in the 1960s, 1970s. And so, we’ve kind of mythologized these things. But they were tiny, they were tiny relative to what’s happening today.

      John: So, there’s a lot more happening today, to the extent to which I can’t imagine a start-up, kind of, thinking about the length and the amount of money that was invested to build the Alto. I mean that’s a major, major undertaking by any measure. On the other hand, I think you’re right. There are now a much larger number of players doing interesting things. And in the software-driven world that we live in, the cost of experimentation and development is not the same amount in terms of capital.

      Sonal: Right, you don’t need that amount of capital anymore.

      Marc: Well, and I agree with all that, but I’d also say — even with what you just said, even that — like, yes, the Alto, but, like, also, like, look, Apple made the iPhone, right? Like, that was, what, a $150 million-dollar project? Like, you know, over the course of its — like, they were able to do that. Google, as you’re well aware, like, basically invented the self-driving car. Those are on par with the Alto.

      John: I mean, if you look at the self-driving car, the tipping point was when the DARPA Grand Challenge was won. And that really was a key tipping point, because it demonstrated the technology was considerable. Considering that the previous contest before that, the car had not driven very far at all, and all of a sudden boom. So, there’s a tipping point in that. And when you see those tipping points, that probably is the time when you say, “Let’s move it from an academic setting that’s kind of more freewheeling, and operates more incrementally, to a different environment.”

      Sonal: Well, one could argue, in that example, that DARPA was a VC.

      John: They were, they were.

      Sonal: Because they were putting up the prize money and everyone was competing in the start-ups, i.e. the individual people trying to meet the challenge, etc.

      John: Yeah.

      Marc: But then Google has now put another, what, dozen years?

      John: Oh, yeah.

      Marc: And a lot more money behind it. And I think that, you know, the self-driving car, the Waymo project, is as glorious a success as anything that ever came out of Bell Labs or ever came out of IBM Research.

      John: Yeah. I mean I think the gap between, “Okay, we can drive on this desert road in a fairly constrained environment,” to, “I can drive in a city environment, with lots of people who do wrong things,” including look at their cell phone while they’re driving, is a much harder environment to do it in.

      Martin: I think another interesting example is a company that you sit on the board of, which is Cisco Systems. Which is, Cisco has long had this stated goal of no internal research. However, they really made modern networking in, like, no small sense of the word, right? <inaudible> in the universities. But when you actually go in Cisco and see what they’re actually doing, you’re like, “Wow, they understand the real problems, they understand the customer.” Like, so I think, like, actually they’ve taken a stance against research there, yet they’ve done a tremendous amount of innovation. However, they have done a good job collaborating. So, it’s a little bit of a spectrum in Cisco.

      John: Yeah. And they’ve had a model for many years of, “We buy interesting companies, and we bring technology in that way, and then we grow it and use the rest of our ability to really make it successful.” So, it’s a different innovation model, as opposed to one that’s more organic.

      Sonal: I mean why wouldn’t you? Because then you’re essentially betting on 1,000 experiments and figuring out which one is a winner, instead of trying to internally, captively figure it out yourself. Like, I just can’t see any alternative to that.

      John: Well, the only downside is that once that company gets far enough along, that little start-up, that it’s got some great technology — there are often more than one company that’s bidding for it. Then you could actually lose out in that setting.

      Sonal: Right, right. You don’t want to lose that. Right, right.

      Marc: God bless America.

      John: Good for the entrepreneurs.

      Martin: I mean it’s actually a really interesting point. The thing I’ve been most impressed with Cisco over the years is, they really, I think, are probably the top company in making those acquisitions successful and doing spin-ins. I mean there are very, very few companies you can put in that have been so successful in acquisitions. So, it’s basically a core competency.

      John: Yeah, it has been a core competency.

      Sonal: “Spin-in” as in?

      Martin: So “spin-in” is, they’ll take an internal team, they will take them out of the company, they’ll help fund them, and then they’ll bring them back into the company once the product…

      Sonal: Fascinating.

      Martin: Yeah, yeah, yeah.

      Sonal: I didn’t realize that.

      Martin: It really has kept them relevant, where many companies have actually not — you know, of the same vintage are no longer.

      John: It has, and it’s injected new technology and new products into the space and things.

      New talent in Silicon Valley

      Sonal: Right. Last question. What do you think has changed with talent — like the whole talent landscape, over the last 30 years? Because we’ve talked a lot about tech trends changing, the availability of capital, the ecosystem, industry, collaboration, academia, etc. But the people themselves in this ecosystem, what is the biggest change that you’ve seen, or are they the same?

      John: So, one of the changes I’ve seen recently, that really has me delighted, is to see the number of young women going into computer science. What’s funny about it is, computer science in the ’80s was one of the…

      Sonal: Yeah, it was very female-dominated.

      John: It was, there were a lot of women in it. And then it got wiped out with the growth of the field and the number of males grew. And now we’ve seen a resurgence — I think, begun by a group of very energetic women that started to build support groups and things like that. And then we got over the critical mass, you got enough women in the discipline that they didn’t feel isolated anymore, and that’s really great to see. The number of opportunities in the software space are so large, we need to bring as much talent in.

      The other thing that’s been remarkable for me is, I thought 10 years ago that computer science was going to become second to the biological sciences, in terms of getting the best students, and that everybody — the really best students were going to go do the biological, biotech, things like this. Well, that’s changed. And now computer science gets the very best students in any of these fields. I mean, I’ve seen freshmen that know more mathematics than I knew when I was a senior getting my college degree now. That’s remarkable, and they’re going to build great things, I believe.

      Sonal: And those are merging, actually. Like a lot of the comp. sci. folks are now starting bio start-ups.

      John: Yes, they are, they are. And bringing computer science knowledge to the bio space.

      Sonal: Yeah. Do you guys have thoughts on any big talent shift you’ve seen?

      Marc: I think the big one I see, that I think is probably under-remarked on, is engineers are so much more productive today, especially in software, than they were 20, 30 years ago. The tools are so much more sophisticated and powerful than all the infrastructure technologies. And then all the — the ability to learn. Kind of to your point on the undergrads, but, like, the ability to go online and learn. Right? It’s like, I’m an engineer and I don’t know how to do something, like, I don’t have…

      Sonal: Stack Overflow it.

      Marc: Boom, boom, boom. I know it in 10 seconds.

      John: Yeah. You may actually be able to find the piece of code, because code sharing has become such a big part of what we do, reuse.

      Sonal: Right, right, right. I mean MIT was a pioneer there with the MIT license and open source. What’s your biggest shift?

      Martin: I think the biggest shift that maybe has impacted me is, like, I just remember the transition where pretty much everybody was in computer science for the love of it, because it wasn’t really clear where the industry was going. Often they were doing it to get something else done — to basically the professionalization of an industry. Meaning, it is a real discipline, people are in it to make money, people are in it for a future. Which is not a bad thing, it’s just required. And I think it’s actually quite good, because it requires us to really think about what it is, what people do.

      And so, kind of on the negative spectrum there’s a — you know, people are lot more mercenary about it than they were before. And on the positive end, I do think we have a lot of framing around it — what does it mean to have a workforce in computer science that will come and go, and to handle that in a way. But for me, it’s been a very, very stark difference to people that I used to work with 20 years ago, when we were literally all there, you know, for the love of solving these great problems — to now it’s like, you know, this is your job.

      Sonal: I think my favorite thing is seeing the intersection of art and humanities and code. And people used to keep them as separate in their heads, and there’s a whole new wave of talent that’s native in both. And that’s really exciting to me because, you know, art is code, code is art. So, to me that’s, like, the biggest, or most exciting, talent shift.

      Well, John, just want to say thank you for joining the “a16z Podcast.”

      John: Thank you, delighted to be here.

      Martin: Thank you very much.

      Marc: Cool. Thank you, John.

      • John Hennessy

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      How to Manage a PR Agency

      Shannon (Stubo) Brayton, Margit Wennmachers, and Sonal Chokshi

      One of the company building topics that’s surprisingly mystifying is PR — and only surprising since so much of the strategy and tactics behind public relations are actually hidden from public view. We’ve tried demystifying the topic in an ongoing series, covering everything from “the why, how, and when” of PR” and leaders building a personal brand to crisis communications.

      But the most frequently asked question startup founders, especially technical ones, have is how to manage a PR agency — from when to bring one in and the mechanics of onboarding and engaging with them; to key acronyms to know in the process of doing so (what’s an AoR? RFP? GA?); to what are the ideal configurations for the who-what-where of in-house vs. agency PR.

      So this episode of the a16z Podcast provides perspectives from both sides of the table (in-house vs. agency, big company vs. startup) for what it takes, featuring PR legends and veterans Shannon (Stubo) Brayton, chief marketing officer at LinkedIn (formerly at OpenTable and formerly vice president of corporate communications at eBay) and Margit Wennmachers, operating partner at Andreessen Horowitz who heads up the marketing function (and who co-founded and later sold The Outcast Agency), in conversation with Sonal Chokshi. It’s not dictation — whether from company to agency, or agency to reporter, or PR to internal stakeholders — there’s a lot of strategic thinking involved even with seemingly incidental things. And… it’s a leap of faith.

      Show Notes

      • Definition of PR and the advantages of working with a dedicated agency [1:37]
      • Deciding when a business needs a PR agency [8:47] and various specialties [16:04]
      • Common terminology [20:39] and further discussion of agency types [26:07]
      • How to work with an agency [27:45] and how to choose the right one [31:45]
      • Advice around PR during and after an IPO [34:36]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I am Sonal. We talk about everything from tech trends to company building on this podcast, but one of the company building topics that’s surprisingly mystifying is PR, since so much of the strategy and tactics behind public relations are actually hidden from public view. So today, we cover the frequently asked question of how to manage a PR agency — from when to bring one in, and the mechanics of onboarding and engaging with them; to key acronyms to know in the process of doing so; to what are the ideal configurations for the who, what, and where of in-house versus agency PR.

      Joining us to have this conversation, we have Margit Wennmachers, operating partner at Andreessen Horowitz, who oversees a16z’s marketing function — which includes brand and communications, among many other things. Before that, she co-founded and sold The OutCast Agency, which worked with some of the now biggest internet companies — then startups — and continues to work with leading tech companies today. And then we have Shannon Brayton, who is the chief marketing officer at LinkedIn. She has worked in-house in corporate communications at several prominent internet companies, including OpenTable —  which later IPOed — and formerly served as vice president of corporate communications at eBay. Together, Shannon and Margit provide perspectives from both sides of the table — in-house versus agency, big company versus startup — for what it takes to manage PR.

      By the way, you can find more background and other topics touched on in this episode, including PR 101, crisis communications, and building a personal brand, in our podcast series. You can look under the public relations tag. But this conversation begins by quickly recapping the “why” of PR. The first voice you’ll hear after mine is Shannon’s, followed by Margit’s.

      What PR is and why it’s important

      Sonal: So how do you guys actually answer the question of, do they even need an agency or not in the first place?

      Shannon: Well, I always start with, “What are you trying to achieve?” So, if you are there because you are trying to get users, PR is not the only lever you should be pulling, obviously. But if you are trying to get funding, or you’re trying to get acquired, or you’re trying to get someone else’s attention, PR is a good lever to pull, but you don’t necessarily need an agency to do that. <Right.> So it all depends on the objective.

      Margit: Yeah. And if it’s early stage, PR can be very helpful, because it makes everything else easier, right? So it makes hiring easier. It makes fundraising easier. It makes everything easier, but there’s almost never a direct correlation. I mean, it’s like, I did this interview in Fortune, and therefore, I got these customers. It’s just not how it works.

      Sonal: So you’re saying it’s valuable, but there’s not necessarily a direct immediate…

      Margit: It’s hard to measure…

      Shannon: It’s a leap of faith.

      Margit: …on a spreadsheet.

      Sonal: Right.

      Shannon: It’s a leap of faith.

      Margit: Yeah, exactly.

      Shannon: I think the biggest tip I give people that ask me about this is, don’t expect to hire an agency and get results immediately, just, like, all of a sudden. You actually need to invest in the relationship both ways. The closer I’ve brought an agency, the better the results. The more I keep them at an arm’s length, the results just completely go downhill, and then the expectations and the relationship just starts to sour.

      Sonal: But I have a question about this, because honestly, like, if I’m a founder and I’m busy trying to build a freaking company, for god’s sake — and in the beginning, you’re doing like 20 things at once, wearing 20 hats — I don’t have the time to manage an agency. Like, isn’t the point of outsourcing it to just not think about it?

      Margit: An agency is ideally managed not by the founder, because as a CEO, you’ve lots of things to do, right? Like, you’re growing [the] company, you’re probably managing engineering still. Like, there’s just a lot of things to do. So ideally, you have someone with a lot of input from you who will help the agency be successful, and the other way around. And so, yes, you do weekly phone calls, and you share lists, and like, this, that, and the other, but it’s like, okay — building camaraderie and, sort of, the investment together. The agency can only tell the stories that they know, right? So if you are the founder that has that gene and understands, like, “Oh, this will be interesting to WIRED. That will be interesting to TechCrunch. This is not interesting at all.”

      Sonal: Which almost nobody can do.

      Margit: Like, that’s just — this is not a given that you have that skill. So, then, if you don’t, then you need to let that team in so that they can discover the stories and tell them for you. And then they do all of the story-finding, the fact-finding, the backup — like, who needs to tell the story, and all of that. And then they figure out how to map a particular idea to a particular person at a particular outlet who will be interested in that.

      Shannon: And building the story is not, “Oh, hi, I’m gonna call you, and I’m gonna tell you the story, and then you’re gonna tell the reporter.” Bringing the person into your world and letting them help find it — experience your culture, get to know you a little bit better — that’s gonna make the story much richer.

      Margit: Yeah, it’s not dictation. There has to be a story. There has to be some tension. Who is well suited to tell the story? What kind of proof point do you need, if any, right? So there was a dance.

      Shannon: That’s, sort of, tip number one, right? Is, you have to be willing to invest in the relationship.

      Sonal: …to invest that fund in the relationship. So there’s a real dance I’m hearing, but quite frankly, what I’m also hearing a little bit of — and just disillusion me of this — why would I even bother outsourcing this? Why not just hire someone in-house? Like, why do all this work if you’re saying it’s not dictation, you should be able to discover…

      Margit: It’s totally legit. I mean, like, I don’t think Apple has really used PR agencies. There are models where you don’t have an agency at all. What we should do is dissect, like, how would you divide and conquer? Like, who is good at what? If you’re internal, you probably are — more of your time is often consumed with talking to the product managers and figuring out what the roadmap is, right, and doing all that. Finding the stories, making sure that everything moves forward, doing all those meetings, right? And you don’t have any of that when you’re on the agency side, right? Your time is taken up with talking to reporters all the time because you have multiple clients. And as a result, you know stories that are kind of outside of scope. And that’s interesting to a client. Whereas, like, you know, it’d be good for me to know if WIRED is doing a cover story on AI.

      Sonal: So you’re saying, an agency, in that context — that person has the time to really keep their tabs on what the reporters are doing all the time, essentially.

      Shannon: Two things I think when it works really, really well. So one is when you’ve got way more program dollars to spend, and you’d rather do that than bring on a headcount, which is much more of a fixed cost. So a CFO, actually, a conversation with the CFO is typically, “Oh, an agency sounds like a great model instead of hiring three people that may be very, very hard to manage, and then potentially have to exit at some point.” Number two, if you get the phone call, “Hey, we’re gonna launch in India, but we’re actually not gonna put an office there or a country manager, and we just need some arms and legs on the ground,” that works beautifully, to be able to find an agency in the network, and call up and say, “We really need your help with project A, and if it works out, we’d love to keep you on.”

      Sonal: So that’s more like more specialty-type things.

      Margit: Yeah. You can dial up and down. And so, particularly, you know, once you’re a really established, high-growth company, you wanna grow the team, you want all of that. But, like, when you’re not in that stage, an agency can be super, super effective.

      Sonal: So I hear the thing on — that specialties may vary, and that some of the media relationships may vary. But are you also, guys, saying then that people should outsource their media relationships to their agency? Or do the in-house people also still invest? Like, how do you, sort of, divvy that bit up?

      Shannon: I think it really depends on who’s got the relationship, and there should not be pride of ownership.

      Margit: No.

      Shannon: If the agency has the better relationship — like in Margit’s case at OutCast, they probably had way better relationships with reporters than half their clients. The client has to be willing to say, “Margit, I’m fine with you calling Quentin Hardy. I don’t need to do that.” And not having that conflict of interest between the two, it should really be whoever’s got the best relationship.

      Sonal: Right.

      Margit: And at the same time, I think it’s really important for internal folks to be talking to the press. Because you kinda lose touch. What is even a good story? What are they thinking? Like, what’s kind of in the water? And not knowing that at all is just really detrimental. However, do you need to be booking all of the appointments, or do you need to be working on every story? Probably not, right? So it’s a balance.

      Shannon: And the internal team, too, gets a lot of G-2 on other companies by talking to reporters directly, too, if they build the relationships.

      Sonal: What’s G-2? Sorry.

      Shannon: Some information about other companies. What’s happening in the Valley, in the industry, just intel, in general. And so I think you don’t wanna ever have the agency team do all of it at the exclusion of the in-house team. They should have their own.

      Sonal: It’s like you’re basically not outsourcing your insights and your connections, but you are trying to scale what you do, and leverage where you don’t have specialties, etc.

      Shannon: Right. Without potentially fixed costs.

      Margit: I remember being in situations where the reporter won’t tell the company directly, like, what’s up, but they’ll tell you.

      Sonal: You’re the inconvenient messenger.

      Shannon: Or like a conduit.

      Margit: And then on the client side sometimes, sometimes it’s really nice to have an agency walk in and deliver a particular message, versus you do.

      Sonal: I’ve actually heard the best definition of consulting, in general, is that they’re the people who will say what the internal people think all along but just don’t get across.

      Shannon: This is why I love letting agencies do media training, too, because agencies can really tell a CEO, “Actually, you look like a complete dweeb,” or, “This message completely did not resonate,” where the employee sometimes has a harder time landing that message.

      Overview of PR specialties

      Sonal: So, we’ve been talking so far about common configurations for working with an agency — like, having it in-house and an agency. Let’s switch and talk about timing. When should you bring an agency in or not? Now, one assumption — maybe this is a good assumption, I’ve heard — is that a lot of times, in the very early days of a startup, at least — you don’t have any hires, you just need to launch an announcement out. Is that the time to bring an agency in, or should you be trying to hire?

      Shannon: I think an agency, to come in at the very beginning and talk about the company’s narrative, and what are your value props, what are your market fit, and who are the people running this company — super, super valuable to have an agency help you do that, without having to hire someone at the outset.

      Margit: I totally agree. One thing that I’ve learned in working with a lot of startups is that you need to, like, strip all the language that we are used to, because they can’t hear it. So, for example, a CEO will call and say, like, “We’re launching soon. We need to have an agency, you know. Like, can you arrange the launch?” And then, oftentimes, people say, like, “Well, first thing we need to do is we need a messaging positioning, blah, blah, blah,” and they just hear, like, “Blah, blah, blah, dollars, blah, blah, blah, dollars,” and none of it means anything to you. Try to work backwards from where the CEO is. Okay. So, when is the product ready? You know, it’s all different, whether it’s consumer enterprise, right, like, and work back from there and go, like, “Well, if you wanna have your launch date be that, that means you’d have to be on the road doing interviews then, and then before that.” Just kinda meet them where they are, and try to strip out all the technical lingo out of the thing, because that’s like engineer talk.

      Sonal: It goes back to starting with what the goal is.

      Shannon: Right. And one of the things I’ll ask people, too, at the outset is, what is your desired headline? Let’s think about your dream cover story. What does that say, and what is the headline? And then you get a really good view into what’s going on when you ask about the absolute dreaded headline, because you can actually glean what is potentially underneath that you’re not necessarily being told.

      Sonal: Isn’t that a little bit dangerous also for them, though, in terms of mixed messages? Because a lot of funders, especially those who haven’t done communications and marketing before, they assume that, “Oh, I’m gonna say this. This is the truth. This is the message. There’s no variation from this.” It’s like logic, not story, which has multiple flavors of interpretation.

      Margit: And so what you would do is, like, you basically are taking them on the journey with you, right? If they say, like, “I want the headline to be, like, we have the best search engine technology on the planet.” And you’re like, “Okay. Well, then, how — like, what would the reporter need to come to that conclusion, right? And so then you walk — again, you walk backwards with them where you have them go substantiate the story. And then, if you can’t, then you’re discovering that together, right?

      Shannon: You don’t commit to the headline, but you basically say, “If this is the type of story you want, here’s all the stuff I’m gonna need to be true in order to help you go for that.”

      Margit: And don’t trust an agency, ever, that will promise you a headline or a story or any kind of outcome.

      Sonal: So how does the life cycle of a company look? And if you could set up an ideal model, is there an ideal model?

      Margit: So, like, an example — you’re basically an established person with an established career in Silicon Valley, and all you wanna do is, you know, put out something that you’re working on something, so you get an inflow of, like, talent and, you know, investors knocking on the door, and whatnot. And you just wanna be very efficient about it. You probably have a relationship with a reporter, and you pick up the phone, he’ll assign it to someone. You do this one story and off you go. And then you go back into stealth, and you’re done. Like, you don’t really need an agency.

      Sonal: Then what would the next inflection point be to bring in an agency again, or to have one on a regular basis?

      Shannon: I’d say a huge influx of media inquiries coming to you, because then you realize, “Actually, this is getting out there, and we’re not controlling the story.” So I think if you’re getting, you know, 5 to 10 a month, you’re probably at a point where you may need an agency to help you manage through that.

      Sonal: So when you get a lot of media inquiries, other inflection points.

      Margit: There are companies that, like, really don’t have any profile, and they’re thinking of going public. I would say, you have left money on the table if you find yourself in that position. Because there’s not that much you can do. Like, once you meet with bankers, you’re on a quiet period, so you can’t really be doing anything, right? So, if you are a company where there’s product-market fit, the company is growing, things are going well, you’re thinking of — if it’s an enterprise software company, another office somewhere — you have a management team, right, and you’re thinking like, “You know what, we should get the CFO in the door, because in the next two years, we wanna be in a position that, if the market is good, we might file.” You definitely need to get a PR firm in yesterday, because you only have a very short window in which you can do anything at all before you go quiet again.

      Sonal: That’s actually rather counterintuitive, because I would think that you would want PR when things aren’t going very well, but it sounds like you’re saying, actually, when you have traction but a low profile, right.

      Margit: You want a different kind of PR.

      Shannon: That’s right. And that’s the next inflection point I was gonna say, which is — it’s more incidental than inflection point, but a crisis —I would not leave it to a founder or a management team to manage a crisis on their own if they don’t have deep PR expertise.

      Margit: You’re not allowed to do this alone.

      Sonal: And frankly, even if you did, I was about to say, I’ve seen these guys in action. Like, there’s a diagnostic thing that only a crisis person can come in and do.

      Shannon: Absolutely.

      Margit: You want professional help.

      Shannon: The best example, the trivia one — they desperately needed some PR help way earlier than they probably actually got it.

      Sonal: Oh, right. This is — you’re referencing the story where the founder emailed the PR folks and said, “Don’t quote this guy. I have this article.” And that person who was working for them kinda went rogue and did their own thing. I remember that.

      Shannon: Correct. And they were all over social media, and the whole story was just going in a million different directions.

      Margit: Nobody was in charge over there.

      Shannon: Boy, they sure could have brought in some help. One other thing, too — I think there are certain people I’ve worked with who think, “Oh, I don’t need to tell her about that thing.” It’s like some skeleton that I inevitably end up finding out about. And so I wish I knew upfront, because I would potentially do things differently. In year one, if I knew that in year two it was gonna come out, you had a DUI or something.

      Margit: The thing is that, you know, I think lawyers — I’ve never been in this situation, knock on wood, but I think lawyers tell you, like, “I don’t wanna know if you did the thing or whatever.”

      Shannon: Right. PR people do.

      Margit: I wanna know.

      Sonal: It’s the exact opposite, that’s right.

      Margit: Because I cannot help you if you’re lying to me, and I cannot steer the ship story-wise if I don’t know what to stay away from.

      Sonal: No information is privileged for a PR person.

      Shannon: That’s right. And by the way, with my DUI example, it’s totally hypothetical, but I would not put a founder into a car for a photoshoot if I knew he had a DUI.

      Margit: That’s a great way…

      Shannon: Because I know that picture is gonna be amazing for reporters to use in a year. So I need to know that.

      Margit: But that’s a very different muscle, right? So there’s the kind of PR that helps you build, right? And then there’s the kind of PR that helps you protect and, you know, figure out the most elegant way to be in a crouch.

      Sonal: Proactive and reactive, basically, essentially.

      Margit: Yeah. Well, there’s reactive stuff where, you know, things are going great, and people want to write about you, and you…

      Shannon: And you wanna take advantage of the building phase, yeah.

      Margit: And you react to that. But, like, all of that is building, right? You are trying to see stories that don’t exist, right? You’re in a crisis, you’re trying to minimize a story that very much exists.

      Shannon: A very certain type of person enjoys managing through a crisis. And so you wanna get an agency that has that type of person, or that type of deep experience in the trenches.

      Margit: Its wartime people.

      Sonal: This goes back to the whole notion of specialties, which you both mentioned earlier. So, are there any other specialties on the PR? I’m sure there’s a million flavors, but like…

      Margit: Yeah. There are people who are really good at product reviews. And if you’re a hardware company, and like, product reviews are — they’re their own animal, right? Like, and you wanna make them, for the most part, as my belief, really high touch, you know. Someone is there to hold the reporter’s hands — obviously, proverbially, you know, as they discover the product. That’s its own muscle. There’s a set of people who will review products. You know, if you do it right, you know, you even get coverage about, like, the unpacking, the whole nine. But it’s, like, that’s very different from exec comms.

      Sonal: So, so far, I’ve heard exec comms, product, I’ve heard crisis. Are there any other flavors?

      Shannon: Developer, like, dealing with developer communities.

      Margit: Consumer.

      Sonal: Consumer.

      Shannon: “I could do “GMA,’ ‘Today Show,’ or ‘CBS This Morning,’ which segment do you want?” That’s a real specialty, and those are usually people based in New York who have deep relationships with the producers at these shows. For a consumer-facing product, that’s the nirvana — is to have a “Today Show” slot, right?

      Margit: Still, even though…

      Shannon: Still, which is hard to believe. All these years.

      Margit: I just saw this morning, apparently, in 2019, people will get the majority of their news online and not on TV. I’m like, “It hasn’t happened yet?”

      Shannon: That’s surprising.

      Sonal: Well, that’s where the reality does come in. I have a friend who founded a beauty-related company, and she was saying that getting a placement in “InStyle” magazine — and also, incidentally, based in New York — was the only thing that moved product for them.

      Margit: And the thing is, like, you do the holiday gift guide press tour in July, and you do the little stint, and you do giveaways, and it’s beautiful photography, and it’s all that stuff. It’s, like, it has nothing to do with, like, how you blog for a developer.

      Shannon: I worked at eBay for seven years, and so every July, we literally had, like, a Santa in 100-degree weather passing out eBay-related gifts, literally, in Times Square.

      Sonal: That’s hilarious.

      Shannon: Because that’s when you had to get in the gift guides, was in July — to make it to December.

      Sonal: That’s so great.

      Shannon: But you need someone that knows that. You need someone that really understands how to execute that type of event, and you need that kind of lead time.

      Margit: Yeah. I mean, like, literally, I’ve come across companies where they call in November, and they wanna be in the holiday gift guides.

      Sonal: Oh, that’s too late.

      Margit: And that’ll be the following year.

      Sonal: On the magazine side, like, the thing’s already shipped. Like, you’re already late, physically, let alone editorially.

      Margit: You know, if you’re a product company, like, how would you possibly know?

      Sonal: Right, exactly. Are there other milestones like that? So, as we’re on this topic of managing an agency — clearly, part of this process — you guys began with the need to discover the stories internally. You’ve talked about proactive and reactive moments, incidental and building ones. Now, what are the other kind of broad milestones to think about in a company’s life cycle that could be newsy moments that an agency could get involved in?

      Margit: So there’s, like, you know, we exist, right? And let me just say, a lot of them you bundle together, right, like, “We exist,” and that’s interesting depending on who the people are and what they’re working on, all of that. Then you’ve raised money. And of course, whether that’s an inflection point, the bar for press coverage keeps raising because, you know, people raise rather large rounds these days, right? Then you get a rock star independent board member. All of those are ingredients, right? Okay, product, right, like, and what are the product milestones? And then you have product-market fit where, all of a sudden, 50% of Wall Street uses your product. Like, there’s lots of milestones. And I’m not saying every one of them is a big to-do, but they’re markers.

      Shannon: And then you wanna handle them strategically.

      Margit: I’m so glad you brought this up. I think a lot of people, particularly when it’s their first time out, they’re lucky that this PR thing happens. They have a nice product, and it’s viral, and this and the other. And they don’t understand, like, how lucky they are, and they can’t separate the two. And then they just think it’s just rinse-and-repeat without doing anything. And those are flukes, right? Most of the rest of us, you need to be very deliberate, and thoughtful, and smart about, like, what moments you aggregate together or can go stand alone, who cares about the different things…

      Shannon: What’s the vehicle.

      Margit: …and what’s the consolation.

      Sonal: What do you mean by what’s the vehicle?

      Shannon: Sometimes you wanna put a tweet out. Sometimes you actually wanna just give it to a reporter exclusively. Sometimes you wanna write a press release.

      Sonal: Do people still do those?

      Shannon: It kinda depends on what you’re trying to achieve.

      Sonal: So what are other milestones? I mean, you mentioned earlier the gift guide. Were there any other consumer-type ones like that?

      Shannon: There’s a company I’m talking to right now who’s based in L.A., and they wanna go national. So, if you’re really taking your story out of a small area and making it national…

      Margit: That’s an excellent place for an agency because, you know, if you are launching — if you’re one of those, like, companies in the transportation bucket, right, like, you’re not gonna put people everywhere, but you are gonna launch in different markets, right?

      Sonal: Right.

      Shannon: For sure.

      Terminology and categories

      Sonal: So we’re kinda getting local people, players on the ground. So let’s shift gear here and just go deep for a couple of minutes on some terminology. I love doing this on the podcast. We do taxonomies and terminology.

      Shannon: And people mostly don’t like to admit that they don’t know a lot of these PR terms.

      Sonal: I know.

      Shannon: They think PR is fluffy and, “Oh, I should know it.”

      Sonal: Right.

      Shannon: But actually, when you dig into it, you realize most people don’t know most of those phrases.

      Sonal: Even I didn’t.

      Shannon: So this will be useful.

      Sonal: I was on the flip side of receiving PR pitches, and when I first came here, someone mentioned GA, and I was like, “What the fuck is GA?” And it’s generally available for the product being available. Let’s do a little buzzword bingo. So AOR.

      Shannon: Agency of record.

      Sonal: And what does it mean? Why does it matter?

      Shannon: It means that you have a master agreement with an agency who’s going to service your business for a certain amount of money every single month.

      Sonal: So it’s like a retainer.

      Shannon: It’s a retainer. It’s a contract.

      Margit: So let me ask you a question. Because you’ve been on the corporate side, so like, you have an agency of record, but you have multiple agencies.

      Sonal: Oh, that’s interesting.

      Shannon: Okay, so agency of record would be…

      Margit: Is that a bragging right, or what is?

      Shannon: It’s probably the biggest piece of your budget. It’s the deepest relationship. And, say it’s an agency who covers everything except crisis, and then you have a crisis. You would go outside of your AOR to supplement their skill set.

      Margit: With another agency.

      Shannon: With a different agency.

      Sonal: RFP.

      Margit: Request for proposal.

      Shannon: Request for proposal.

      Sonal: And?

      Shannon: Should we do this at the same time?

      Margit: Oh, my god. So that’s sort of — oftentimes, companies use this process where they put out an RFP to multiple agencies so that they have some, sort of, standard set of questions and answers, right? And oftentimes, the bigger the company gets, the more involved they can get, and they’re often driven by procurement and not the actual PR team, although they’re obviously orchestrated by that. And it’s sort of, like, who are you as a business, right, like, what are your revenues, team size, locations, all that kind of stuff, right? And then how would you handle X? Or give us your ideas for Y. And then the agency comes in — and oftentimes, you send in the questionnaire, and then there’s sort of this bake-off where you do the proposal, and you bring the team.

      Shannon: It’s a dog and pony show.

      Margit: It’s a dog and pony show. You come up with really good ideas based on nothing, and then you have to sell. I’m not sure I would recommend them. I don’t think they’re the most effective thing, because it comes down to, oftentimes, where like, that agency has really good ideas…

      Shannon: The big thing I would add on RFP, too, is do not bring in the shiny object from D.C., or the shiny object from New York, if that is not your account team. Bring the people who are going to work on the account.

      Margit: Oh, absolutely.

      Shannon: That is one thing an agency will do, is like, “Oh, John is here from D.C.”

      Margit: Our expert from…

      Shannon: “He was Clinton’s social media policy person.” And then you never see John ever again. He was there to win the pitch. When I would do RFPs back in the day, I would always tell people, “I only wanna see the team that I’m gonna be working with every day.” It’s really important.

      Margit: That’s super important. And that’s why, you know, if you actually do the process that way and you make that requirement, that team is ready to go.

      Sonal: You’ve literally onboarded them, actually.

      Shannon: They’re in it. Yes. So I’d love to get your perspective on this. If you put your account up for review because you’re not happy with your agency, the incumbent agency should not be allowed to pitch. I think it’s terrible for the agency, and I think it’s terrible for the company.

      Margit: I’m gonna say something harsh. If the agency actually wants to repitch the business, you should fire them again.

      Sonal: Why?

      Margit: Because it’s just a stupid-ass decision. Replace the revenue, learn from — why is the account gone, and then move the hell on.

      Shannon: I could not agree more. You set up all these bad dynamics by doing it.

      Sonal: Well, I do have a question, though, which is — how do you avoid the complacency problem when you do have an ongoing agency of record, and you wanna light a fire under them?

      Margit: So I think OutCast was really good at this because we had clients like salesforce.com, for example. We had, I think, about a dozen years from 1998 onwards, right — through, I think, 5 VPs of marketing before the company filed for IPO. And that was, like, Caryn Marooney. She just did an amazing, amazing job managing that account through changes, inflection points, growth. And what we would do is, we would ask ourselves, we would ask the teams, like, “Okay, so what’s on the to-do list, and what’s not on the to-do list?” And, like, forget what the client is telling you during the weekly calls. Like, what do you think will blow them away? And then go do that. And, like, do not ignore media, because if there’s a good story, you have permission to do everything else. You can have every conversation. You can screw up occasionally. But if there’s no media, then, like, what are you doing? And as a result, we always were called a media shop. And I was like, “You know what, I’ll wear that badge with pride, because I think it’s important.”

      Sonal: What does it mean to be a media shop? I actually don’t know what that means.

      Margit: That the only thing that you can do is talk to reporters.

      Shannon: Or the thing you do best is media relations.

      Margit: Right. Look, you’re not really good at crisis, you’re not really good at building out, like, messaging — but like, that’s the thing. It’s, like, you get so mired into, like, “This is what’s on the to-do list.” And mind you, if you’re working with a company, your client may be sophisticated, and they may not be. So you need to do that, like — you need to have your own motor and machine to go, like, “Okay, what should it look like?” And then go do that.

      Shannon: And on my end, I would say, at eBay, the best relationship I ever had with an agency was with Capital Communications, who is based in New York. And the reason that we were able to avoid complacency with them is we were basically categorized into different product categories. Basically, they had clothing, shoes, and accessories, but it was like, “If you nail this, you can get another category.” So we kept upping the pie. I mean, we’re not fit for every single category, right, but it was always a possibility. And so that kept them from being complacent.

      Sonal: Besides the domains you guys have described — like, consumer, enterprise, some of these pieces of the pie — or specialties like crisis, local — what are your thoughts on the other competencies within a shop? Like, should you go for a one-size-fits-all agency that offers interactive and writing, and other things? Should you go for a boutique firm that only focuses on X? Like, do you have thoughts on the size and the types of the agencies themselves?

      Shannon: I don’t mean to sound like a broken record, but it depends on what you’re trying to solve for.

      Sonal: Well, that’s a fair point.

      Margit: Sort of very broadly, right? If the company is doing well, basically across the board, and it’s global, it’s kinda nice to have an AOR that is global, because you can just turn knobs, you can turn things on and off, and then you can add and layer in on the step that you need.

      Sonal: You mean like a global agreement.

      Margit: Yes, a global agreement because, you know, if you’re that established, it’s nice to go like, “Okay, I need the Indian people to do X, right? I need the team in South Africa to lay low for a while.” That’s easier. Otherwise, you have to figure out, “Who’s my international agency there? And who manages…?”

      Shannon: I do think a lot of agencies promise that they’re amazing at that, “Oh, don’t worry, my Greece counterpart will easily talk to the Brazil counterpart.” It doesn’t happen as seamlessly as it looks, so you really have to dig before you hire them on for a global job to make sure that these processes are really down.

      Margit: And you will often give up the higher a bar for the global.

      Shannon: For the coverage, yeah. It’s true.

      Margit: Because I don’t think, like, nobody can tell me that there is an — I have not seen an agency that is equally excellent in every office that they have.

      Shannon: Absolutely.

      Margit: I just don’t.

      Working with an agency

      Sonal: And let’s spend a few minutes now just talking about some mechanics of engaging with an agency. We’ve already talked about the process of why it matters, the terms, you know, the configurations. Give me some more of the nitty-gritty of bringing them into your machinery. So, you guys mentioned, like, you wanna have regular check-ins. Should they be weekly? Should they be daily? Should there be someone in your office embedded? Should there be someone just on a phone call? How do you manage all that?

      Shannon: So, weekly, usually, has worked for me in the past. I think embedding them — those are the people who have done the best with the company. They can walk around, they’ve got great relationships, they get a vibe for the culture, they can go to an all-hands meeting and hear the story from the CEO. There was a model, again, with Kaplow, where they had somebody that went on leave for six weeks, and I went to New York and worked out of their office for six weeks. And that was amazing for me, too, to get to know the agency.

      Sonal: You inverted it, literally.

      Shannon: Yeah, I inverted it. And I got a better view of how the agency ran, and what they needed more from my team, and how challenging the time zone thing was. And it was really valuable.

      Margit: There’s nothing that, sort of, is like picking up the phone and going like, “Hey.” It doesn’t have to be, “I’m reporting back to you on what I promised.” It can be, “I just had coffee with a reporter, and they’re really interested in X,” or you know, “Your competition seems to be all over…” like, you know. So just sharing intel back and forth, because you need to communicate enough so that you’re in a groove. It’s a little like talking to our kids. Like, you need to be talking — basically, you need to be in some communication all the time, so that there’s a groove. Because you want to be top of mind for the agency so they’ll pay attention to you, and for the agency, you want to be top of mind so they’ll include you in stuff.

      Shannon: One thing we did at LinkedIn, too, is — quarterly, we would sit down with the agency, and we basically had, like, a red, yellow, green on the things that we had agreed they would do, and talk about what was working, what wasn’t. And that was an opportunity not for us to just say, “Hey, this isn’t working.” They were able to tell us, “We can make this work if we had more of X, Y, and Z.”

      Sonal: So you set them up for success.

      Shannon: Exactly. And it felt like you could have a transparent conversation without hurting the other person’s feelings.

      Margit: So I will add one more thing. I actually think that it is important for the CEO to know at least some people at the agency, and for them, occasionally, to staff a meeting. And a lot of people probably disagree with me on that.

      Shannon: I totally agree with you.

      Margit: Because one is, eventually, there’s a bill, right? And like, if they don’t ever see you, right, it’s, like…

      Shannon: Out of sight, out of mind.

      Margit: “What am I paying for? What is this?” And also, there’s a reason that person is the CEO and the founder of the company. They are the originator of the idea. They are, hopefully, the most compelling person to talk about what it is and where it is going. And again, if you want to enable a bunch of storytelling, then you ought to have the people have access to the holder of the story.

      Shannon: The in-house PR person needs to give up a little of the control to allow the agency in sometimes.

      Sonal: This is actually one of the questions I wanted to ask, which is — a founder/CEO should be pretty — not too involved and not uninvolved.

      Margit: The thing with, you know — if you are the founder of a company, you’re trusting those people to represent your soul to the outside world, which then gets written up in some way by a neutral third party — hopefully neutral. I think that those are people you wanna know.

      Sonal: Absolutely.

      Margit: Don’t you think?

      Shannon: Yeah, I do, too. And I think a lot of CEOs don’t have the patience, because they, again, have this mindset, “I just wanna outsource it. Isn’t the whole point of me paying for all this so I don’t have to deal with it? You’re taking care of it,” etc., but you’re saying that they are — it’s like sales. You need to put the CEO in front of the account.

      Margit: Yeah. I mean, look, if you don’t wanna deal with PR, then don’t bother hiring an agency. A normal reporter — completely reasonable reporter — will expect to see and meet and talk to the CEO.

      Sonal: So more on the mechanics side, tell me — we were talking about the RFPs or the proposals, and getting all these. And whether you do it or not, you clearly may have multiple people you’re considering as potential agencies. How do you vet them and know that they’re the right one for you?

      Shannon: I think the best agency referrals come from reporters. So, they’ll come in and they’ll pitch, and you’ll say, “Wow, that team was amazing. Great ideas.” But to get to the real heart of who they are as media relations experts, you call reporters you have relationships with and say, “What do you think of so and so at such and such agency?”

      Margit: I love you for saying that, because I will tell you, so many people, when I say, “Oh, you’re referencing them with reporters?” they’re like, “Oh, good idea.” Duh.

      Shannon: Duh.

      Margit: That’s their customer.

      Shannon: So referencing them, for sure, and then other clients. Either clients that have churned out — you get a lot of feedback that way — or current clients too who are probably relatively happy, you think.

      Sonal: What if the problem is on your side as a company? What if the turnover is on your side, and you’re the reason the agency is not doing well? How do you sort of tease apart the truth and the reality? Does it even matter?

      Margit: It’s excellent, because many companies don’t even ask themselves that question. So they’ll call, and they’ll go, like, “We’re unhappy with our PR firm.” And then you start digging, like, “Why, what’s going on?” Well, they’re just not executing. And you just try to diagnose, “Okay, so do they know what to do? Are you giving them information? Are they part of…” you know. And something I always say was like, you know, “If you’re a good client, they have a chance. If you’re not a good client, then they don’t even have a chance.”

      Shannon: That’s why I like that report card thing, because you can easily say, “Look, you’re not executing on this,” and they can say, “Well, you didn’t give me access,” or “You didn’t return my call.”

      Sonal: You mean the red, green, yellow you were talking about.

      Shannon: You use that review to basically talk about what you, as a company, could be doing better. And you’re absolutely right, it’s usually two-sided. It’s very infrequently just the agency. It’s just much easier to blame the agency.

      Margit: Right. Also, my favorite is, like, new CMO comes in. First thing they do is do an agency review. Happens all the time. But, like, I think it’s just so much smarter to take some time and go, like, “Okay, what’s actually going on here?”

      Sonal: So that’s about diagnosing the success of it. How do you know it’s successful, it’s working, that your partnership — this is not a measurement question, because I know we’ve talked about a lot and there’s not a clear-cut answer. How about selling people internally? What if you, the person who’s managing the agency, knows it’s working — but your CFO who holds, like, the purse controller doesn’t agree, or how do you make that case internally?

      Margit: Well, you obviously try to explain and educate, like, what it is and what they’re supposed to do. And then, at the end of the day, it comes down to — do you have a mandate? I think it’s good to include everybody in the process and in the updates, have them see and engage with the agency, and do all of those things, right? But at the end of the day, to me, it’s a little binary. You either have a mandate to figure out how to make your function successful within a certain budget parameter, or you don’t.

      Shannon: And it’s making sure, too — you can’t be assured that the CFO saw the big story in “The Chronicle” this morning. You actually do need to do some promotion of the work that the agency and the team do together.

      Margit: PR it back internally, basically.

      Shannon: Exactly. You can just expect everyone’s gonna fall into your story, or absolutely saw it. So you actually have to do a little promotion internally.

      PR during an IPO

      Sonal: So we talked about some major inflection points in the company. What about going public? So, you talked about, Margit, earlier — about the importance of having PR milestones in place before you go public. So, Shannon, you’ve taken a couple of companies public. What’s some special advice you have for those thinking about that particular piece?

      Shannon: So, a financial comms agency is super, super helpful in the lead-up to your first earnings call. So there’s the actual IPO piece, which I think a lot of people think is much trickier than it really is. The day of is a very busy, crazy day, but it’s not rocket science, necessarily. I think getting an agency to help write a script, and get the CEO and CFO to really be on the same page about the story and practice it — it’s a nerve-racking day.

      Sonal: For the roadshow?

      Shannon: No, your first earnings call post-IPO.

      Sonal: Right.

      Shannon: So, I think the IPO has this misnomer that it’s, like, super challenging and tricky —and yeah, there’s some complexity to it, but what’s really hard is your first earnings call as a public company. And so that’s where I would optimize for getting an agency in to really help figure out, “Are these two on the same page? What is our story? Where are we, and where are the gaps?”

      Sonal: And does that person report into an investor relations function? What if that doesn’t exist yet?

      Margit: It depends. I’ve seen investor relations report into the CFO, and the comms person reports, sometimes, to the CEO or CMO.

      Sonal: And, Margit, of all the startups you’ve seen, what would you say is the best reporting structure for where an agency and the comms person should, kind of, report into? Is there any variations on that?

      Margit: So, I am a proponent of having the PR person report to the CEO. It can report into the CMO. But oftentimes — so if the CMO is a quant person, I think definitely PR should not report into the CMO, because it’s just a different animal. If the CMO is a brand story type of person, it can certainly work. Regardless of who does your review, the PR person needs to have access to the CEO whenever they need to.

      Shannon: And then that layer in between, whoever it is, cannot be upset when the PR person goes direct to the CEO and doesn’t necessarily loop into them. So they have to have — everyone’s gotta be okay with the fact that, “I may not report to the CEO, but I have direct access, and I don’t need to bring you in every single time I talk to him or her.”

      Sonal: It sounds like the CEO is the one responsible for setting that precedent and culture, that all these pieces are okay.

      Margit: It’s telling, right? When a CEO refuses to do that, the caliber of talent they can get just drops 10 notches.

      Shannon: And it shows, too, that they may not value the function as much as we would like them to.

      Sonal: Okay. You guys have given us a lot of food for thought in terms of how to concretely manage an agency, think about it strategically. What are the things that you wish you could tell founders and, like, hit them over the head with every day — that you’re, “I want these guys to know this?”

      Margit: I think founders have an idea that you call a reporter and go, “Hey, can you write this story?” It does not work that way. And I wish people were a little more thoughtful about the function — the comms function — and actually how strategic it needs to be and how well thought out it needs to be before you take a story to an important reporter. It’s not easy.

      Shannon: I was gonna argue almost the flip of that, which is, it seems like there’s a new breed of founders who, sort of, are in this internet-native world of “blogging is everything.” Go direct. You don’t need any middleman. Eliminate all the middlemen.

      Margit: You can do that, but you will never be as credible as an independent source of information. And you may also not have the eyeballs that you can get when WIRED writes a story, and then it gets shared on social channels. Worse, let’s say you’re established, you’ve done fine on your own, things are great. Well, what if something goes sideways? You have no relationships, you have no trust, you have no benefit of the doubt. All you have is — people, like, hear all of your claims and adoration, and all of a sudden, you have to do a product recall. Wouldn’t you like to have some relationships in place where at least you can explain yourself? I think so.

      Sonal: Kind of points in the bank, too. That’s great. Any other final common myths, misconceptions, parting…

      Margit: I would just say, you know, as much care as you take with your product and the hiring of your first 10 engineers — and like, whose money you take, right? Take as much care to decide, like — you’re either going to take this function seriously, and then actually do it and learn it, because there are people who are there to help you — or actually don’t bother. Because spending $2 with a half-hearted commitment is a complete waste.

      Shannon: And don’t be afraid to admit when you don’t know something related to PR.

      Sonal: Hopefully, that’s the point in this podcast. Thank you, Shannon and Margit, for joining the “a16z Podcast.”

      • Shannon (Stubo) Brayton

      • Margit Wennmachers is the head of marketing and content at a16z, where she also advises entrepreneurs on their communications and marketing strategies. Previously, Margit cofounded the The OutCast Agency.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Technological Trends, Financial Capital, and the Dynamics of Disruption

      Fred Wilson and Chris Dixon

      There’s all sorts of interesting tech trends happening right now, including AI, VR/AR, self-driving cars and drones (as well as interesting stuff happening in verticals like healthcare and finance) — and there’s a lot also happening in seemingly more “mature” tech revolutions, such as mobile and cloud. But where are we now, really, with these shifts… and how does that inform how we think about the next couple decades?

      And does a framework like Carlota Perez’s — as outlined in Technological Revolutions and Financial Capital: The Dynamics of Bubbles and Golden Ages and summarized by venture capitalist and longtime internet investor Fred Wilson (of Union Square Ventures) — fully apply when it comes to software? Because, argues Chris Dixon (general partner on a16z crypto), software “has so much more plasticity, ability to adapt, ability to evolve” that unlike hardware, “the core itself will also dramatically change… not just the apps around it”. The total economic value that will be unlocked with the software revolution, observes Wilson, should be orders of magnitude bigger than what we saw with manufacturing for sure.

      But just how much internet innovation is actually powering true disruption (i.e., is more than just a sustaining innovation, to use Clayton Christensen’s terminology)? How do new business models change everything? Dixon and Wilson consider all this and more in this hallway-style episode of the a16z Podcast, where we recorded the two having a think-aloud conversation about everything from the history of the internet and startups, the evolution of capital and infrastructure, to the advent of crypto. How do they they both define “decentralized”, what do they think of dApps, and where do NFTs and “crypto goods” come in?? One thing’s for sure: It’s the most interesting time they’ve both ever seen in over 30 years of internet work, life, and play.

      Please note that the a16z crypto fund is a separate legal entity managed by CNK Capital Management, L.L.C. (“CNK”), a registered investor advisor with the Securities and Exchange Commission. a16z crypto is legally independent and operationally separate from the Andreessen Horowitz family of fund and AH Capital Management, L.L.C. (“AHCM”). 

      In any case, the content provided here is for informational purposes only, and does NOT constitute an offer or solicitation to purchase any investment solution or a recommendation to buy or sell a security; nor it is to be taken as legal, business, investment, or tax advice. In fact, none of the information in this or other content on a16zcrypto.com should be relied on in any manner as advice. You should consult your own advisers as to legal, business, tax and other related matters concerning any investment.

      Furthermore, the content is not directed to any investor or potential investor, and may not be used or relied upon in evaluating the merits of any investment and must not be taken as a basis for any investment decision. No investment in any fund advised by CNK or AHCM may be made prior to receipt of definitive offering documentation and due diligence materials. Finally, views expressed are those of the individual a16z crypto personnel quoted therein and are not the views of CNK, AHCM, or their respective affiliates. 

      Please see https://a16zcrypto.com/disclosures/ and https://a16zcrypto.com/disclaimers/ for further information.

      Show Notes

      • Discussion of the early internet and how it drove new technology [1:20]
      • The current state of technology, including cloud and SaaS [17:31]
      • Changes in the current capital market [26:21] and the enormous potential of crypto-based technology [37:51]
      • Discussion of timing around tech developments on the horizon [49:30]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. We recently took the “a16z Podcast” on the road to New York City. And so, today’s episode features a fly on the wall recording of a conversation between venture capitalists and longtime internet investor, Fred Wilson of Union Square ventures, and a16z crypto general partner, Chris Dixon. The two discuss everything from current tech trends like AI, to seemingly more mature trends like mobile and cloud to the advent of crypto.

      And throughout the episode, they weave through past and present in internet history to reflect on where we are now, really, to where we may go next. Are we seeing disruption? Or is a lot of current innovation just “sustaining innovation,” to borrow from Clay Christensen’s framework? Or, how about Carlota Perez’s framework for tech revolution and financial capital? Does that fully apply to software? And what, precisely, is so unique about software itself? And how might that affect internet native applications for things like crypto?

      The two discuss all this and much more in this episode of the “a16z Podcast.” As a reminder, the content provided here is for informational purposes only, and does not constitute an offer or solicitation to purchase any investment solution or a recommendation to buy or sell a security, nor is it to be taken as legal business investment or tax advice. Please also see a16zcrypto.com/disclosures for further information there.

      Early internet and tech developments

      Fred: So, 2018, you know, I don’t know what it is now — 25 years into the internet, sort of the modern — the mainstream internet. Some very large, some, you know, trillion-dollar tech incumbents, very — venture industry has grown dramatically. Tech seemed more mainstream. Some people argue it’s, perhaps, becoming mature, you know, maybe it’s like TV where there’s like four TV channels, and there’s four big incumbents. There’s all sorts of interesting stuff happening in, like crypto and AI, and new devices like virtual reality and augmented reality and self-driving cars and drones. There’s interesting stuff happening in verticals, so, like, changes in healthcare and real estate, in finance. You know, so I guess — I don’t know, a question I think about a lot and, I’m sure you do, is where are we now? And how does that inform how we think about the next decade, two decades?

      Chris: So, you know, I think it’s really important to have a framework. And as a lot of people who, you know, follow me and USV know, we’re really big fans of Carlota Perez. And, if you look at the big technological shifts that have happened over the past 200, 300 years, what you see is that many of them last for, kind of, a century. Like automotive, or more broadly speaking, industrial or manufacturing-based businesses were, kind of, the technological revolution of the 20th century.

      And, even though internet and tech, kind of, emerged very powerfully in the ’80s, and ’90s, I think we might argue that internet and tech really is the technological revolution of the century we’re in now. And so, if that’s the case, it’s 2018 — we have 82 more years to go and yet, like, we’re feeling like it’s already a mature sector. We’re looking at everything, Google, Amazon, Facebook, Apple — you know, they have these massive entrenched platforms, and it’s becoming really, really hard to compete with them. How could we have, you know, another 50, 60, 70 years to go?

      And I think, you know, going back to something that Marc Andreessen said, you know, in the original design of the internet and the World Wide Web, you know, we didn’t necessarily get it all right. And so, what we have now, in terms of, like, these entrenched platforms, is maybe a function of what the spec was that we’ve been building on for the past 25 or 30 years. And, if we could maybe broaden out what the protocols are, to a point where, you know, we could have a much more open and level playing field, we might actually have a lot more ways to go. So, that’s me speaking optimistically about where we are.

      Fred: Let me give you maybe a more — even a more optimistic view, which is — so, one thing about Carlota Perez, and I think it’s a brilliant, you know, brilliant theory and explains a lot of history. But I wonder if software is different than hardware. And let me try to explain that. So, you build a car, you know, 1900-ish, you build cars, you have Ford and GM, etc. They get better but, you know, they don’t — fundamentally the car has stayed mostly the same — you know, obviously gotten better, but they’re still the same basic function, same basic, you know, kind of, design.

      And then, all the action from, I don’t know, 19, probably what, 30s to today moved to, kind of, the “app layer.” It was [the] interstate highway system and shopping centers and shopping centers and trucking systems, and the sort of the “apps of cars.” And then TV, like the TV, you build the TV, you have the radio, you know, you build the physical infrastructure, and then maybe you upgrade color, or HD or [something], but it’s fundamentally fixed. It’s fundamentally — once you’ve built the core, it’s fundamentally done, and then you just build around it.

      The Internet, as you, of course, know, is deliberately designed to be a very lightweight protocol, where all of the intelligence lies at the nodes and the servers, and are software-based, and can upgrade themselves. And so, I guess what I would say is, I wonder if software, because it’s fundamentally a software — I mean, of course, at the bottom level, it’s copper, and other, you know, fiber and other things, and radio waves. But the heart of it is software. And, I just wonder if software has — my view, software has so many more degrees of freedom. It has so much more, kind of, plasticity, ability to adapt, ability to evolve, that maybe unlike the car and the TV, that the core itself will also dramatically change, and not just the apps around.

      Chris: So, I think, certainly the total economic value that will be unlocked with the software revolution should be at least one, maybe two, maybe three orders of magnitude bigger than what we saw with manufacturing, automotive, you know, for the reasons you described. I just think that it’s just, if everything, if we can get everything into software, then just imagine, like, what the possibilities are.

      Fred: It’s the expressive power the — it’s the design, it’s a much richer design space. You can write it, you can do so many you can encode —any kind of human process can be encoded in software.

      Chris: Yeah, I mean, even, you know, the greatest thing about Tesla is the over-the-air software upgrade. It’s like, literally, I get in the car and all of a sudden, I got a new upgrade, and there’s, you know, new software, all of a sudden.

      Fred: And they actually fixed like the breaking and stuff where…

      Chris: They can fix anything. But I read today that they pushed an update to people who have cars in the affected areas of Hurricane Florence, I don’t know — to make their batteries last longer, like so that they could go long without charging. That’s fucking amazing. Just think about that, boom. Kind of, imagine like Elon Musk woke up thinking he was gonna do that and he just hits a button.

      Fred: Well, and it has implications like so, if I remember reading a book about Ford and the whole — you know, the big problem was they broke down all the time. And so, the big innovation was service centers everywhere. And so, you know, this change in being able to update over the air also changes, for example, the service model, and, therefore, the, kind of, almost, you know, the industry structure, potentially.

      Chris: I mean, the car, I mean, that car is still not a piece of software, but there’s more software in that car than all the cars that I’ve been driving for my entire life. And so, you know, we’ll get more and more of what we interface with in business and our personal life, whatever, that’s gonna be software as opposed to hardware. And I just think that that makes the total available market of this technological shift way, way, way larger. I think we’re in a moment in time where it feels like there’s not a lot of innovation. And, you know, I will admit that, you know, the past, maybe four or five…

      Fred: Well, are you saying innovation or disruption?

      Chris: Disruption is a better word.

      Fred: Yeah, there’s a lot of innovation, right?

      Chris: Yeah.

      Fred: I mean, there’s great stuff happening in AI and self-driving cars, and…

      Chris: Right, but a lot of that’s accruing to…AI is a really good example. So, you know, if you look at the big tech companies, I don’t think that they were disrupted very much by the shift from web to mobile. And, I don’t think they’re gonna be disrupted very much by the adoption of AI power software.

      Fred: It may just, it may further entrench their monopolies because of the data — the fact that you’re differentiating the AI through data, and they’re most likely to have the most data.

      Chris: I forget where I read this, this data is now four or five years old. But if you look at the top 100 mobile apps, and you look at what the top 100 websites were before the iPhone came out, it’s not a very different list, right? So, you know, those companies — Facebook, I think, is the perfect example of this — saw that the mobile phone and mobile apps could potentially disrupt them. They pivoted their focus to get there, and they got there. And the net of it is that they are in a stronger position than ever. So, I don’t think, you know, that a lot of the innovation is currently powering a lot of disruption.

      Fred: One thing that mobile did, you know, you take the ride-hailing as an example. It did enable new behaviors which allowed for startup opportunities that — I mean, so it did — you know, I think of it as every new computing platform has new capabilities. And, generally, startups will exploit the uniquely new capabilities, and incumbents may or may not successfully port over to the platform. In the case of mobile, they did. But it also unlocked, you know, so you had the intimacy of the camera allowed for Snapchat and…

      Chris: But even that, you know, I’ve always thought that the maps layer could commoditize the entire ride-sharing business. It hasn’t, like Uber and Lyft, and a bunch of other companies, and a bunch of companies in Asia dominate that business. But I don’t understand why the map on your phone, whether it’s an iPhone or an Android phone, isn’t the ride-sharing application. And then all of these ride-sharing networks…

      Fred: And it’s maybe what Google wants to do longer term. And one would think with their self-driving car effort and the maps…

      Chris: It just seems like…

      Fred: You’ve got a button — you use Google Maps, you open it up, you’ve got a button and get a car and…

      Chris: Like, why does that happen today? I don’t get it like how — I mean, I don’t understand why the map interface hasn’t become the default interface for dispatch of anything. Whether it’s a scooter, or a car, or a bike, or whatever — maybe it’s just, like, it’s gonna take some time. Maybe that stuff doesn’t happen overnight. But that’s where I wanna do dispatch. So, even that, I think, you know, the mobile OSes have the opportunity to suck that functionality in. And that is one thing that’s going on a lot right now, is that the functionality is getting sucked more and more into the operating systems and the proprietary apps that these big companies have built on top of those operating systems.

      Fred: And so, when you say there isn’t a lot of innovation, I think what we’re — to clarify, there’s a lot of clever, you know, inventions happening. There’s AI. I think of it as, like, AI, new computing platforms, crypto, etc. But there’s a lot of that happening, but a lot of it feels sustaining, as, like, Clay Christensen would say, not disruptive, right?

      Chris: For sure.

      Fred: So, it will reinforce the current industry structure, not change it.

      Chris: I think that’s right. But yet, you know, and this is why we’re so interested in crypto. I think crypto is the one innovation out there that feels highly disruptive, because it’s a real change in business model. It’s not just a technological change. It’s a fundamental change in business model. You’re not monetizing with ads, you’re not monetizing with subscriptions, you’re monetizing with the underlying token. It’s, like, orthogonal too.

      Fred: So mobile wasn’t. Mobile was, some would say, the standard. The internet was disruptive. So in some ways, crypto, like the internet, is the first, kind of, potentially major disruptive wave. Is that your view?

      Chris: Yeah, I mean, I think what Google did is that they made advertising the business model for applications. Like, my mail and, you know, my browsing and my searching is all supported by advertising. Like, Microsoft would have made me pay for that. They would have said, you’re gonna pay for Explorer, you’re gonna pay for Outlook, and that’s how we’re going to monetize it. Google comes along and says, “No, that’s an ad product.” And so, that’s what’s disruptive, is, like, that’s just — everything changes when you change the business model. It’s like, all of a sudden, you know, your strengths become your weaknesses. And, that’s what allows a lot of new entrants to come in.

      Fred: It’s funny because I talk to people now who I think weren’t around during the early internet. And a lot of people will say — I hear it a lot — that, well, the internet came along and, you know, by ’94, had all these killer apps. And, you know, I think one fun exercise is to go watch movies from the ’90s as a way to, kind of, go back and look at it. And, first of all, the interesting thing, the internet just doesn’t exist. There’s no mobile phones. Every once in a while, someone will, like, it’s like this ritual. They’ll say, “I’m gonna go online,” and like the very phrase, like, go online, kind of is this, kind of, archaic phrase that is from that era. And then suddenly there’s, like, this beeping and there’s this and that, and then there’s like, you’ve got mail, and they get on and they do this thing, and then they get off, and like it’s this thing, and it’s the thing you do for 10 minutes.

      But it almost — if you actually watch movies from the ’90s like the internet, even that doesn’t happen except for, like, hackers or a couple of, like, specialty movies or something. And it was essentially, you know, I mean, it was like literally waiting for an image to load, and, like, innovations of the time were things like having the image load in a less annoying way because it was that slow.

      Chris: Like, I remember listening to what we would now call podcasts over dial-up in ’97, ’98. A friend of mine, Josh Harris, had this company called Pseudo, and he was making basically audio and video.

      Fred: Wasn’t it kind of like “Justin.tv,” like, he’s filming his life and…

      Chris: Yeah, yeah. He was just 15 years too early, but he had all the right ideas. And, you know…

      Fred: All the ideas that happened, Webvan, Instacart — like, they all, all of them — I had this game, this board game, I still have it somewhere, it was called “Dot Bomb.” And it was, like, making fun of all the terrible ideas. I blogged about this once, and it was, like, this joke game about all the stupid ideas in the ’90s. And it’s literally, like, every unicorn, like, hot company today. It was, like, internet money, you know, grocery delivery.

      Chris: Well, you know, you and I are friendly with an entrepreneur here in New York named Adi Seidman. Adi Seidman had YouTube in like ’99. It’s just like it didn’t make any sense in ’99. I mean…

      Fred: Yeah. Well, it’s broadband, he needed the infrastructure. Like, I mean, I think it was 2000 — was YouTube 2005, I think?

      Chris: ’05, ’06.

      Fred: Like, that was kind of the moment when broadband really tipped and you could plausibly have — I mean, it’s a model — the internet wasn’t a real thing, I think, until you had broadband.

      Chris: I also think we needed — I think the two big moves that made YouTube successful, whereas all the people who tried YouTube before YouTube — were broadband, that’s, like, 80% to 90% of it, and also social sharing. It’s the idea that you could take a YouTube video and embed it on your MySpace page, or even just send somebody, like, a URL, and boom, they hit it, and they could watch something. Like, the idea that everybody was going to go somewhere to watch something with social and social sharing, it blew that idea up. Now, all of a sudden — the embeddable YouTube player, like, that was genius.

      Fred: Yeah. Before that, I remember it was like heavy.com and they’re really like, there’s a whole concept before that — was destination, it was business model. Part of it was a business model, in addition to your earlier point. Because the thought was you need to get eyeballs on your page, stickiness, these concepts, which prevented people from encouraging, why distribute — like, the whole idea of embed, like, one school of thought would have been letting people embed YouTube — well, how do you make money? How do you do this? But that was one of the very insightful things that YouTube creators did.

      Chris: Right. But broadband, I think, you know, was the main thing, but everybody had that opportunity. Everybody who was trying to do YouTube at that time was benefiting from broadband. So, there were a couple things that YouTube did that others didn’t do. And I think the embeddable player was maybe the move that differentiated them from everybody else. And just maybe the timing. Plus they had a lot of venture capital money behind them. So, they could spend money. Sometimes that’s the key difference.

      Fred: Yeah. I think I was reading this history of — I think it was Vimeo and a few of the other competitors. And, they had just a whole different set of financial constraints and couldn’t — and also, frankly, the view towards copyright. I think YouTube took a more laissez-faire view.

      Chris: Right. But that ultimately caused them to have to sell to Google, because they concluded that they couldn’t continue to play the game that way, which was the right way to play the game as an independent company. They needed somebody who could fight the content owners.

      Current state of technology

      Fred: Yeah. Okay, so going back to what’s happening today then. So, kind of, the big — I mean, I think there’s multiple layers here. So, there’s, kind of, the big core tech trends. I don’t know if you agree, but I think it’s, kind of, AI, kind of, proliferation of new, kind of, computing devices.

      Chris: I still think cloud, I think cloud is still — I mean, cloud’s like a 10-year-old-story. But I still think cloud is driving a lot of innovation. I still think…

      Fred: Cloud infrastructure or apps, SaaS or…

      Chris: Well, I’m thinking infrastructure. 

      Fred: It’s pretty amazing how having the fresh codebase, having the fresh attitude, the different perspective, you can kind of type them.

      Chris: If you could build on Stripe versus you couldn’t build on Stripe, you know? Like, that was a big difference. Like, that whole — I mean…

      Fred: Well, just the whole modern thing and as a developer, I think the whole concept of, like, developer experience, that really thinking through, you know, I think you didn’t — obviously, you were an investor in Twilio. One of their big focuses was, kind of, the “time to delight” or something, or time — “hello world” of just, like, immediately getting in there.

      Chris: Like, the idea of introducing text messaging into your app, like, what’s the big deal about having to be able to text natively from an app. But you think about it, and, like, so many little things — but they’re big things — are enabled by an app being able to text you. Like, with a two-factor code, or your car’s arriving, or whatever, like — and to do that before Twilio was like, hard. And, now it’s like five lines of code.

      Fred: But how do you think about — so in that, in the cloud infrastructure world, you have AWS, Azure, Google Cloud.

      Chris: That, I think, is game over, mostly. Like, maybe there’s somebody who’s gonna come out from left field.

      Fred: Game over in the sense of incumbents are winning, or AWS has won it or…

      Chris: I think incumbents. I think Google’s gonna take some shared — I feel like what Google is doing is going top-down into that market. They’re going to some of AWS’s biggest customers, and they’re saying, you know, we’ve got some better tech, maybe they do, maybe they don’t — we’ll do it for less, and we’ll care about you where they don’t. They’re taking share. But I think it could be, like, Amazon has 60% of the market, Google has 30% of the market, and Microsoft has 10% of the market. I think it’s gonna be a three-player game in that business. And that’s good.

      Fred: I think the thing I hear from Google is the challenges. It’s just that, sort of, the high touch concierge enterprise, kind of, model, which those companies, you know, whatever, the Citibanks of the world expect is just not something they’re used to, although they have now they have Diane Greene. In that, I think they’re…

      Chris: So, maybe Microsoft will get there, you know.

      Fred: The data I’ve heard is Microsoft has been making more progress because they’re used to it — because exactly that is — I mean, Microsoft fundamentally is an enterprise company today, right? And they’re very, very good at, sort of, you know, servicing these large corporate clients. And, then they also have, as I understand it, these sort of, like, they tie everything together, effectively a new kind of bundling. You already have Office, you have all this other stuff.

      Chris: But there’s a second kind of enterprise customer. There’s the truly legacy enterprise customers who are, I think you’re right, it’s a very high touch, white glove kind of thing. Then there’s the new enterprise companies, the Spotifys of the world, right? Like, Spotify is gonna either be on Amazon, Google, or Microsoft. But Spotify might not be high touch. Like, they may just want really really good infrastructure.

      Fred: And they may care that, for example, Google seems to have an advantage on, you know, their TPUs, their AI chips, for example, you know, all the, kind of, bells and whistles and fancy technology they have, which, of course, they will have.

      Chris: I think winning the hearts and minds of developers is really hard when you’ve got an incumbent, and it’s, like, the standard. Like, every developer knows how to build on AWS. So, if you’re starting a company, where are you gonna build it on? You’re probably building it on AWS — 90% of the companies, maybe 95% of the companies we back are built on AWS. But once you’re an established company, and you can think about maybe moving from one to another, I do think, — I think that there’ll be three players in that market.

      Fred: So how do startups fit into cloud?

      Chris: Well, I think they’re a big beneficiary as a cloud infrastructure.

      Fred: Okay, right. So, building on top, and you’re able to take what was CapEx and move it to OpEx and not worry and focus on what you wanna focus on — music playing…

      Chris: By the way, that’s a story that has been playing out for the better part of 10 years. What I’m saying is, I think there’s still some legs to that. Like, you were saying, what are the big drivers? It’s AI, it’s AR and BR, it’s crypto. And then we were talking about a few other things that I think are still playing out. And one of the things that’s still playing out is cloud. I think there’s still some legs there.

      Fred: Well, I think also on the app side. I mean, SaaS — I mean, I think we live in this world where we think everything is, you know, is SaaS-based. And the data I see is something — it’s, like, sub-10% of corporate applications today are still — you know, it’s unbelievably low. I mean, people are still using, you know, I don’t know, you go to the whatever, you go to the United desk, and they’ve got like a DOS interface, you go here, and they’ve got Windows. You know, I mean, the government hack was, like, Cobalt, you know.

      And so, and a lot of it’s more modern, but it’s still like, it’s Windows, it’s whatever, it’s not — you know, we live in this, kind of, world where we think everything is SaaS-delivered. And so, you know, you see these really interesting things happening, where, like, vertical SaaS is an interesting trend, where you see this, you know, whatever, massage parlors need their own billing and reservation system or something. And it turns out that’s a big market, and those people previously wouldn’t really have software.

      I, kind of, think of it almost like — people have this view that like AI is gonna come take the jobs, but they have this, kind of, anthropomorphic idea that it’s going to be like the Jetsons, like, this robot is going to come in and like, “Move over, I’m gonna type and do it.” But actually the way that the jobs are actually taken is a much subtler thing, which is these new software applications, just, you know, whatever, your new, you know, payroll system, your new payroll SaaS app, just suddenly, you don’t need as many people in your payroll department.

      And it’s a much subtler thing. And it’s not a one-to-one transition. And it’s not a literal thing of, like, kind of, like ,Elon Musk, AI comes in and, like, replaces your payroll department, it’s just like, more and more just, kind of, incremental software comes in and just makes everyone more efficient, which takes jobs. Now, I think it also will create many jobs, they’ll just be — it’s just always harder to envision the jobs that are created than it is to envision the jobs that are destroyed.

      Chris: What you’re talking about right now, I think is where a lot of the action is in startups and venture capital right now. If I look at where, you know, a lot of the dollars are being invested, where a lot of the value creation is happening, where a lot of the bigger companies are getting built — it is sort of in that enterprise SaaS area. I think consumer has gotten harder because of some of the things we’ve been talking about before, but I don’t think that enterprise SaaS has gotten harder.

      And so, in a way, I think, for — we’ve thought for the past four or five years, it’s more of a grinded out execution, operational, not super sexy style of startup and style of investing that has been winning the day. And the things that were, kind of, happening in the latter part of the last decade, where you had, like, a Facebook come out of nowhere, a Twitter come out of nowhere, or a YouTube come out of nowhere, where VCs were making 100 times their money on these huge big consumer breakouts — we see those but not as many.

      Fred: Yeah, I mean, the great thing about SaaS is the business. Once you have those customers and they like your software, they stick with you. So, it’s like an annuity. It’s a very high margin annuity. It’s a challenge to get those customers, and you have to get — there’s a lot of detailed, kind of, sales and marketing execution to go reach efficiently. And that’s, kind of, the trick in a lot of those companies, is how do you efficiently reach the massage parlor, the whatever, small business payrolls, you know, person who needs payroll.

      Chris: The founders all complained to me, they’re like, every venture capital meeting I take they wanna know, CAC over LTV, they wanna…

      Fred: Yeah, yeah, yeah. It’s very metrics-driven.

      Chris: They wanna know our sales power, they wanna know, like, you know. And I was like, because honestly, that’s what separates the winners from the losers in that world.

      Fred: And in that world, there’s almost always five, or two to five, credible competitors, too. So, it’s like a cage match. You know, it’s not, like, you’re some weird, you know…

      Chris: But they’re also not winner-take-all markets. So, probably still a power-law distribution in the outcomes, but maybe it’s not as severe, maybe it’s not — maybe the winner’s not 10 times bigger than number 2, who’s then 10 times bigger than number 3, which you can see in consumer, but maybe in enterprise, it’s the winner’s 3 times bigger than the next biggest one, who is then 3 times bigger the next biggest one. And if it’s a business that can support, you know, billions of dollars of market value, that could be two or three companies that could be pretty big winners for the founders, and for the VCs who back them.

      Changes in the capital market

      Fred: So let’s maybe talk a little about the changing capital market world. So, the venture world has changed a lot. There’s new ways to fund startups, including, you know, crowdfunding, which you’ve been really involved with. There’s, you know, the ICOs and, kind of, the crypto stuff that’s kind of, you know, emerging. There’s a lot more venture capital, there’s a lot more stages of it, there are these mega-funds. You know, I guess, you know, how have you seen the industry change over time?

      Chris: Well, I think the first thing I would say is, what you said, is that I think there’s more capital available for founders today than there’s ever been. And I think that’s a good thing. You know, a lot of people in the venture business, or the people who give us the money, you know, think there’s too much money, and maybe yes, but the reality is, like, what’s good for founders is good for the venture business. I just, kind of, believe that as, like, a fundamental truth. So, there’s a lot of money out there.

      The traditional angel seed, early-stage VC, then growth VC market — I think the explosion of capital is probably more in the later stage. You know, with the SoftBanks and Sequoia going out and raising however many billions they raised, and so on and so forth. And I think that, if anything, there may have been a slight contraction in seed and angel and early-stage money in the past few years.

      But I think what’s interesting about crypto — so first of all, you know, there’s a lot of people who went and, you know, used tokens, or just a crypto business plan or business model to go tap into like a lot of new money that was showing up in the world of crypto. And I think that may, you know — certainly in 2017, and even now, there’s a lot of that money sloshing around. But I think what we’re really seeing, and I’m an optimist here, and I’m hopeful about this — is that we’re seeing a new capital market being built. We’re seeing a new way of raising capital that’s global from the ground up, that’s not subject to these, I think, antiquated laws around who can invest in startups and who can’t invest in startups.

      And just unleashing the startup capital markets to be global and anyone can play is great. It’s a huge innovation. And I understand there’s people out there saying there’s gonna be scams and, you know, people are gonna invest and lose all their money. And I get all that, and I realize that that’s not all good. But in general, I think that making it possible for everybody to invest in these, you know, high-growth opportunities is good. And I think it’s good for founders that the capital markets, for what they’re doing, are going to continue to grow.

      Fred: I mean, the fact that today you have to be literally a millionaire to invest in startups, to be an accredited investor. If you actually participate in ICO, it’s very complex. You need to download, like, a wallet, you have to, like, put in a long hexadecimal address. I don’t — I mean, I’m pretty, pretty sure that almost all the people that participated in those things were technical people that were into crypto. I mean, the vast majority were, and those are people, like — and I just said, like, anecdotally around Silicon Valley, these were, like, programmers who were very sophisticated, who understood exactly what the technology was, who were maybe not accredited investors, and who felt like now they finally found a market were like they understood the product, you know? I mean, they were customers of the product. They were literally protocols built for developers, and they would use it and they could participate in this. And the idea that, you know, a non-technical, wealthy person is able to invest in that, but this programmer who really deeply understands the protocol is not, just seems strange.

      Chris: Very strange and wrong. The other thing is, you know, if you were an early user of Facebook, you might have said to yourself, “Oh, my God, this thing’s gonna be huge.” It was — so, let’s say that was 2003 or 2004 or 2005, whenever that was — when did Facebook go public, ’11 ’12?

      Fred: ’12.

      Chris: Right. So, it was like, seven years before you could eventually become a shareholder on Facebook. But if you’re an early user of Bitcoin, you had Bitcoin day one. So, what’s great about crypto is that I think it allows the early adopters to not only be users, but also participate financially, if they want to — if they wanna hold on to their Bitcoin, and go along for the ride, they can do that. And that’s why I think that this token business model is so exciting, is that in a way, it kind of takes the world of investing and the world of using and combines them. And like, there’s this — I read this blog post last week about, like, there’s two parts of crypto. There’s the money part of crypto, and there’s the utility part of crypto.

      Fred: “Tech and Money,” I think.

      Chris: Yeah, “Tech and Money,” right. And, like, the money people are, like, Bitcoin maximalists. And they’re really focused on hard money, and the tech people are Etherium maximalists, and they’re focused on, you know, DAP platforms and smart contracts and all that. I don’t buy that at all. That to me is — I mean, I’m not arguing that that’s what the world is today. I think it’s Eric Thornburg who wrote it, and I thought it was very insightful about where we are in crypto right now. But I reject that idea. And I think what’s powerful about crypto is that they’re the same thing — that using and investing are the same thing — and you don’t have to be a user to be an investor, and you don’t have to be an investor to be a user. But if you wanna be, you can be both, and it’s just as easy to be — like, it’s just as easy to be both, right. And that, I think — it’s just not right, particularly for a lot of these businesses where the user is the product. Facebook users made Facebook, Twitter users made Twitter, you know, YouTube users made YouTube. It’s just not right that the people who make the product have no participation in the value appreciation.

      Fred: And I think and beyond that. To me, one of the great tragedies in the history of the technology industry is that what happens is there’s a sort of lifecycle in every network. But I use network in a very broad sense, like, Windows is a network. So it’s a network between developers and users. And there’s a lifecycle, and they start off, and at the beginning, there’s sort of this, you know, the network, the platform wants to attract everybody, and they play very well. And so, you know, they provide tools for this third-party software developer. Facebook early on says, “Hey, media companies, come work with us,” you know, they’re solicitous. And then over time, you know, you hit the, kind of, growth peak, and everyone “it’s a party” and everyone’s having a good time.

      And like, you know, you’re growing 100% year-over-year, you know, you don’t worry too much about stuff. And then eventually the top of the escrow comes, and then it becomes — that’s when things get ugly. And that’s when — and I don’t blame any of the management of these companies or the boards. I think it’s just the logic of the business model. The logic of the business model is, sort of, it forces them at that point, when things start to slow — like Google, like Google search. Like, if you look at mobile, like it’s hard now, you get all sponsored links on mobile, they just keep adding more sponsored links. You have Yelp going up in front of Congress and doing, you know — like, when it’s the growth period, hey, great, we’ll have one little ad at the top and like, but, you know…

      Chris: When the pie is growing, no one’s fighting a piece of the pie. When the pie stops growing, people start fighting over the pieces of pie.

      Fred: And then what happens is these networks have these, kind of, internal battles. And those are the most vicious — like, that’s the funny thing about tech. I think that, like, people outside of tech don’t — so, traditionally, in economics, people think of substitutes as the competitors. So, the hamburger and — this is an example, the hamburger and the hot dog are the competitors. Whereas in tech, actually, the hot dog and the hot dog bun are the ones that have the most vicious fights. Microsoft-Netscape, you know, Facebook-Zynga, like, Twitter and the Twitter apps or whatever, right?

      Because, like with Twitter, it just feels, kind of, tragic to me that you had this giant developer ecosystem that was trying to make the protocol proliferate. And they could have worked together in concert, and it could have been a much bigger and more impactful platform. But the logic of the business model, the ad-based business model, said we need to control the experience, we need to control where the ads appear, we need to, like, etc., etc. Again, not placing blame on anybody. It’s the logic of that model, right?

      Chris: Yeah. Well, look, I mean, you know, it did live through that period. And I was on the board of the company during that time. And I very much wanted to see an open platform. I mean, if you read the blog posts that I wrote when we invested in Twitter, it’s still on usv.com — I wrote about the API and the open ecosystem and all this stuff that was getting built on top of Twitter. That’s what I wanted Twitter to be. But we had an advertising attention-based business model. And the truth of the matter was, there were other people who were out there running around buying up third-party Twitter clients. And inside Twitter, we saw that, and we’re like, we can’t have that. Like, we can’t let somebody go get half of our user base by acquiring all these third-party clients, and then taking them onto a new network, and then we’ve just lost half of our users. Particularly when we have a tension-based business model.

      If Bitcoin and tokens had existed in 2005 instead of 2009, I think it’s very possible that Twitter could have adopted a token-based business model, left the protocol open. Twitter could have been the protocol, and the third-party clients could have been the clients on top of the protocol, and it could have worked beautifully. And, like, part of me just wishes that we could have done that. And then we could have seen how powerful this new business model is. We’re gonna see it, eventually we’re gonna see it, someone’s gonna do it. And we’re gonna be like, “Holy fucking shit.”

      Fred: Well, and the thing you’ll have is, instead of the fighting, you’ll have this beautiful alignment between the core protocol, all of the third-party developers, all of the users. And, by the way, the other thing you didn’t mention when you mentioned the Bitcoin users, not only do those users own Bitcoin and participate in it, they also probably, by the way, disproportionately have Twitter followers, Reddit karma, Hacker News cred, whatever that’s called, you know, Google juice. And they’re out there, like, talking about it and marketing it. And so, you know, you have this army of users, miners, developers, core protocol developers, all fully aligned. And by the way, and control — and probably very powerful, relatively speaking, on the most important marketing medium of, you know, the internet. And it’s just a very powerful combination.

      Chris: I think that’s very —by the way, I think that’s a wonderful thing. I think that’s very scary to a lot of the people whose jobs it is to regulate and protect the capital markets, because they’re looking at that and they’re saying, there’s all these people who own a lot of this thing, and they’re out there promoting the shit out of it. The truth of the matter is, though, it’s — most of these people are not pumping and dumping. The truth of the matter is mostly people are true believers, who have held all the way through. Once my money gets into the crypto ecosystem, it’s gonna stay in the crypto ecosystem. It might, you know, move, like, I’m definitely going to be playing around in all these networks, and, you know, using them and staking and doing all this incredibly cool shit that you can do. But that wealth that I took out of Fiat and put into crypto back in ’11, ’12, ’13, ’14 is not going back that way.

      The potential of crypto

      Fred: Let me just try this out. I have this theory, let’s see what you think of this theory. I have this theory that we are in the middle of a transition period where the digital world is becoming more important to the point —but we’re in a transition period in the sense that we still — it’s even in our — we still think of the offline world as primary. And you see this, by the way, and all the critiques of Bitcoin, for example. Oh, like, in the end, a lot of, like, the, you know, traditional economists were taking — it boils down to, “Oh, it’s fake money. It’s digital, therefore, it’s fake.” And like, for example, like to think that gold, which, you know, has whatever, out of dental and speaker wires, and it has no real utility, like, to think that that has some kind of, like, ontological, like, higher status than a digital good is, I think, evidence of this, sort of, offline bias. And like, and I think you hear it — by the way, I notice this whenever people use language. E-sports, notice — e-commerce — like, when — you have to preface, the online one is the one with the modifier. Right?

      Chris: Right.

      Fred: Right. And so at some point, by the way, these things will flip, and it’ll be, like, that’s commerce, and then there’s offline commerce.

      Chris: You’re saying — the predecessors to Bitcoin were called e-money.

      Fred: Yeah. No, no…

      Chris: And there were a lot of predecessors.

      Fred: And whenever you have a modifier on the thing, you know that people think it’s subordinate. But eventually, it’ll flip and it’ll be, like, Amazon is commerce and Walmart is oh, yeah, it’s like, you know, meatspace, or offline commerce or IRL commerce, whatever they’re gonna call it. So, I think we’re in this transition period. And it’s funny, too, because like, you know, Fortnite made $300 million last month on dance moves, emotes. And so, digital goods have become a massive industry, for example, in video games, and there’s digital resources, domain names — you know, I’ve always bought — I’ve been a longtime collector of domain names. I’m like, let me tell you, it’s a little piece of the internet and like, of course, it’s going to be valuable, and I’ve always wanted them and I hold on to them. And that’s a multi-million…

      Chris: I’ve never sold any of those either, by the way. I never sell a domain name. Why would you?

      Fred: I mean, it’s the — I own a piece of the internet. It’s like owning real estate around Central Park or something.

      Chris: In a way, this is like religion. Like, it’s like, you know, I would never ever sell a domain. I might swap a domain. Like, if you said, you know, “Hey, I’ve got, you know, ABCD,” and I said, “Well, I got…” you know, we might swap, but I’d never sell for dollars.

      Fred: So, I think that when we’re in this transition period where we’re still, kind of, like, anchored on this sort of thing of, like, “Well, it’s not real.” But it’s gonna be obvious 10 to 20 years from now, especially as, like, the newer generation, kind of, grows up. And just, of course, like, of course, an emote is worth more than, you know, offline equivalent or whatever. And that it’s going to be — and these words are gonna — and the language is gonna change with it, and just the whole, kind of, way of thinking about it. And there’s not gonna be this weird thing like, “Oh, it’s digital. It’s not as — ” Do you see what I’m saying though? And I think, like, e-sports is a great… 

      Like, the other thing I think about is, like, a lot of, like — I remember, a lot of times in the history of tech, like, mobile 2011 or ’12, it felt like mobile was definitely growing. But I don’t think, at least for me, until, like, maybe 2012, I didn’t realize it had actually replaced desktop. There was a moment at which it was sort of, like, “Wait a second, this isn’t just like a big thing. This is the thing.” You know what I mean? And you realize, now, in retrospect, that we thought we were in, kind of, this growth period, where we’re actually in this hockey stick. And I think that’s sort of, I believe — one of my theories is we’re in this like — so take e-sports as an example. I think we’re in this hockey stick right now with e-sports. Like, video games, it’s just gonna be obvious that, like, it’s going to seem certain — I mean, I think it’s always great that people play physical sports, I’m not anti, but, you know, it’ll be like horseback riding, vinyl records, like, you know, a whole bunch of other kind of…

      Chris: I think this is where what geeks called NFTs, non-fungible tokens — I think you just call them digital goods.

      Fred: Crypto goods.

      Chris: Crypto goods. I think the innovation here is that these digital assets can be scarce, can be one of a kind. You know, the reality is, like, we’ve never had the ability to make a digital good non-replicable. And that’s I think what, you know, has held back a lot of the business models around digital goods. It’s just like, if you had an mp3, you could give it to a million people. If you had an, you know, an MPEG, you could give it to a million people. So, I was explaining this to  somebody last week, who’s trying to, like, figure out, you know, crypto. And I said, you know, think about all of the digital goods that you earn in a video game, right? And you spend — you know, you’re obsessed about this video game for, like, six months, and you collect all these incredible video — but they’re stuck in the game.

      Like, imagine if you could take them out of the game, and you could put them in your wallet or think about just, like, your bank account. And then, another game comes along, and you could literally take them to another game. And this person who I was talking to, who was a skeptic, like, a fucking skeptic like you wouldn’t believe, he’s like, “Holy shit, my 12-year-old would just be all over that.” Like, that’s like, you have just given my 12-year-old his, like, nirvana.” And I said, but that’s what we’re doing here. Like, that’s where we’re going.

      Fred: I think games will be the first to adopt this stuff, because they just tend to be very, kind of, fast cycle time experimental.

      Chris: But also, no, no, it’s not just that, it’s also who’s playing them.

      Fred: You mean the kids, the kids that are far more open to new technology.

      Chris: Yeah, like, tell a kid to download, like, an Ethereum wallet that has NFT capability, and move their games out of Fortnite into their Coinbase wallet or, like, that’s not gonna get in the way.

      Fred: But I think the other thing with NFTs that excites me is, I feel like this should be a wonderful time in history for creative people with, you know, 4 billion smartphones, and the ability to just sit down and write something and create a piece of music. And from a business model perspective, it hasn’t been. And so one of the things that’s really exciting to me with NFTs is if you — let’s take music as an example. So, mostly on, like — selling the song itself has become a not great business model. And so, a lot of musicians have moved to — they go where the scarcity is, and the scarcity is offline. And so, it’s shows and merchandising.

      And so one of the really exciting things to me about NFTs is the idea that you can reintroduce digital scarcity and have, whatever — exclusive album art, exclusive whatever. Maybe this is — to me, it seems like a very promising new business model not just for games but for writers, for musicians, for, you know, whatever, you know — filmmakers, videomakers. Reintroducing scarcity, allowing, kind of, business model innovation. I mean, game…

      Chris: Well, the cool thing is, it doesn’t have to be that there’s only one of them. It could be there’s a million of them, or in the case of Bitcoin, 21 million. We still haven’t mined all the Bitcoin. And we’re nine years into it now. So, like, imagine if you made a song. I just came up with, I think, a pretty cool idea. Imagine if you made a song, and there was 21 million listens to it, and you had to mine the listens, but it was gonna take maybe 5 or 10 years to mine all those listens. You know, actually, you get 21 million listens pretty quickly on the most popular songs. But like, imagine if there was some mechanism of releasing a piece of art or a piece of music that felt more like mining Bitcoin than it did…

      Fred: You could never do scarcity — there’s no way to ever do — I mean, they tried with DRM and things like that. But this is always…

      Chris: But the point I’m trying to make is like, I don’t think necessarily for the artists the move is super tight scarcity. I think the move might be pretty loose scarcity, but still some scarcity. And the other thing that’s cool that’s happened with Bitcoin is, the value has risen over time. And I know that’s come down recently, but it’s mostly risen over time as we’ve started to mine more and more of it. So, we’ve never seen — well, you have seen that a little bit in art, like certainly physical art, that’s what you do see. But, you know, I don’t have a crystal ball view of how this is all gonna go down. But I definitely think that the move is not…

      One of the things I learned really, like almost, like, painfully, was that when we first started investing in the internet in ’94, ’95, ’96, what we were doing was dumb. We were just basically investing in things that had existed in the offline world, that were getting moved on to the internet. Like, “Okay, so let’s invest in an online newspaper, let’s invest in an online store, let’s invest…” you know, like, but that’s actually not the move. The move is, [you’ve] got to find the native thing that needs to happen now to have this thing.

      Fred: That’s funny. Every form of media, you go back and you look at the early movies, and they were plays, and they were trying to film the play, and then later on, they’re like, “Oh, we can do a close-up, we can do an establishing shot, we can do this, we can do that.” And they just up and develop a new grammar. And then there’s sort of the — there’s the movie-native movies or something. I think it’s a very common pattern throughout.

      Chris: Right, so I think with digital goods, with music, with film, with art, whatever, like, I think…

      Fred: It will be some crypto native thing.

      Chris: I don’t think it’s to make, like, a limited edition piece of your art, like, that there’s gonna be 10 of these. I think it’s something that’s more like the way Bitcoin has happened, where it gets mined over time released by…

      Fred: Something you can never have done before, as opposed to a direct, kind of, whatever — just, like, porting over some concept from the non-crypto world.

      Chris: That’s why I don’t love when entrepreneurs/founders come to us, and they’re like, “We’re gonna take mortgage backed securities and put them on the blockchain, or we’re gonna take — like, they’re really — a lot of people are interested in taking things that exist in the physical world, or the existing financial world, and putting them on a distributed ledger. And I mean, I think that that’s like — I think there is some incremental value to doing that. But what I’m much more excited about is people creating brand new things de novo from scratch, on these crypto networks, that never existed off the crypto networks, couldn’t have existed off the crypto networks, and always will live on the crypto networks. Like, to me, that just seems like a 10x or 100x better idea.

      Fred: Yeah, like DAO is a good example. Like, autonomous organizations. This concept, which is now — it, kind of, fell a little bit into disrepute because of this thing called the DAO, which was, kind of, mismanaged.

      Chris: It’s unfortunate. It’s unfortunate that they named themselves after the big idea.

      Fred: I know, I know. But this idea that you can — I mean, to me, it still blows my mind that you just look at any of these interesting solidity contracts. And it’s code running on the internet. No one — you know, once the developers have shifted out and taken off all, you know, the controlling, kind of, code, it’s just literally code that exists — it’s autonomous code. And it’s its own little organization. And these things will get more and more sophisticated. And this is something that could never have existed before. Like NumArray is one that you guys are involved in, for example.

      Chris: People think that NumArray is a hedge fund powered by a crowdsourced network of data scientists. And it is, but I actually think if you really try to understand what Richard’s doing, I think he’s playing around with staking. Like, staking is a really powerful idea. I mean, it’s existed forever. I mean, if you read Taleb’s new book, it’s really all about skin in the game. It’s about staking. But I think that we’re gonna see a lot of really innovative things being done with staking, because I think crypto-tokens make staking super easy to do.

      And there are — like, I think Richard’s idea is that if I can get data scientists to stake their models, to put skin in the game against their models, I’m gonna get much better models than if they just throw them up against the wall. So, I just think we’re gonna see a lot of innovation around staking, around governance, you know, on-chain governance, is that gonna work? I don’t know. I think, like part of me says it’ll never work. And part of me is, like, I hope it does work.

      Timing of new innovations

      Fred: Obviously, we’re both excited about crypto. How long — and there’s all these great ideas floating around, but it’s early. You know, how long do you think it’ll take to play out — to sort of — I mean, it’ll obviously, hopefully, take many decades, but like, when will we start to see meaningful applications used by people beyond the, kind of, crypto enthusiast?

      Chris: I think that we got to fix — it’s the broadband issue. We’ve got to figure out how to get crypto networks that are truly secure and decentralized, that can handle much higher transaction processing speeds than what we have today.

      Fred: To me, think of the crypto network as a computer. And the computer has different design criteria, it’s very different than traditional computers in-network but in the consensus mechanism, etc., it makes this abstraction that’s a computer on top. And you need trust, you need developer experience, you need scalability. And yeah, it’s like we’re internet pre-broadband, we’re mobile phone pre-iPhone. <crosstalk>

      Chris: Exactly. Like, remember when people were building, like, what were they called? Were they called, was it WAP or something? There was, like, some standard for building applications.

      Fred: I had this poor friend who was, like, into mobile from, like, the ’90s. And I think he, like, gave up in 2006. Like, it’s never gonna happen.

      Chris: That’s what we are. We have not had our iPhone moment in crypto yet. And the right…

      Fred: We need the iPhone moment and then we need…

      Chris: That’s what I’m waiting for. And, you know, it doesn’t have to be totally decentralized. There’s this whole narrative around like, you know, where on the decentralized curve is enough? I just think, like, decentralized means that nobody controls the network. And that’s, kind of, to me, fundamental, like…

      Fred: To me, it means trust, and it goes along with nobody control — like, you trust the network. I trust that if I have this NFT, I really have it. And no one can take it away. And no single person can take it away. No bad system engineering can take it away.

      Chris: What’s the quote, like, power corrupts, absolute power absolutely corrupts, or whatever it is — but like, you just said, like, when Facebook realizes that they’ve got the keys of the castle, they’re gonna start to extract rents. Like, it’s what happens. So, that’s why decentralization is so important. And so, I think we got to have a crypto network, or 2, or 3, or 4, 10, who knows, that is truly decentralized, truly secure, and can process transactions at the speeds — at least of, like, an ATM network or something, you know. We don’t have that yet.

      Fred: Yeah. So, we need that. And then it happens over — so, whenever that happens, and then hopefully we have the 5 to 10-year, kind of, immediate explosion of apps the way that we did with the internet and mobile, and then, a long tail of, kind of, further innovation.

      Chris: Definitely. That’s what I see. And that’s why, largely, we’re not investing in a lot of DAPs. I mean, we are playing around with NFT’s in games, for all the reasons you raise. I just think that’s the first place that we might see it. And, by the way, if you go back and look at the history of infrastructure, sometimes it’s the apps that demand the infrastructure, and then the infrastructure gets there because the apps demand it. So, games, in a way — games might make it such that we get the crypto network we need, right, like…

      Fred: What’s a good historical example where the app — I think actually, by the way, games have driven — for example, games have driven GPUs. So, like, just like, that market is one where the gamers have been endlessly hungry for more polygons. And that created this kind of, you know, Nvidia and this whole industry around it, which then had these interesting —you know, then, of course, spun out the deep learning movement.

      Chris: Well, I mean, it may be a good example, like, a lot of interesting core infrastructure came out of Netscape. Why did Netscape have to build it? Because it didn’t exist.

      Fred: Javascript, cookies, SSL.

      Chris: But if you go all the way back to, like, where we started, I think we are maybe in the most interesting time I’ve ever seen in my 30-year career, but it’s not at the surface. It’s, like, under the water, and you’re not getting rewarded very much as an investor for being, you know, super bullish. Well, last year was great. Like, everybody wanted to own crypto, but I think that was, kind of, like, a wave and it’s certainly come and gone. I don’t think you’re getting rewarded a lot for that. And you’re getting rewarded a lot more for, you know, the more operational execution-oriented stuff, like enterprise SaaS and things like that. But I think what’s super interesting is the stuff that’s gonna start bubbling up and that’s where my head’s at.

      Fred: Awesome. All right. Thank you.

      • Fred Wilson

      • Chris Dixon is a general partner at a16z, where he leads the crypto/ web3 funds. Previously, Chris was cofounder & CEO of startups SiteAdvisor and Hunch (acquired by eBay); and an early blogger at cdixon.org.

      Seeing into the Future — Making Decisions, Telling Stories

      Steven Johnson, Chris Dixon, and Sonal Chokshi

      There’s a lot of research and writing out there on “thinking fast” — the short-term, gut, instinctual decisions we make, biases we have, and heuristics we use — but what about for “thinking slow” — the long-term decisions we make that both take longer to deliberate and have longer spans of impact on our lives… and the world? Because we’re not only talking about decisions like who to marry (or whether to move) here; we’re also talking about decisions that impact future generations in ways we as a species never considered (or could consider) before.

      But… why bother, if these decisions are so complex, with competing value systems, countless interacting variables, and unforeseeable second- and third-order effects? We can’t predict the future, so why try? Well, while there’s no crystal ball that allows you to see clearly into the future, we can certainly try to ensure better outcomes than merely flipping a coin, argues author Steven B. Johnson in his new book, Farsighted: How We Make the Decisions That Matter Most.

      Especially because the hardest choices are the most consequential, he observes, yet we know so little about how to get them right. So in this episode of the a16z Podcast, Johnson shares with a16z crypto general partner Chris Dixon and a16z’s Sonal Chokshi specific strategies — beyond good old-fashioned pro/con lists and post-mortems — for modeling the deliberative tactics of expert decision-makers (and not just oil-company scenario planners, but also storytellers). The decisions we’re talking about here aren’t just about individual lives and businesses — whether launching a new product feature or deciding where to innovate next — they’re also about even bigger and bolder things like how to fix the internet, or what message to send aliens with outcomes spanning centuries far into the future. But that’s where the power of story comes in again.

      Show Notes

      • Discussion of what long-term decision-making means [0:24] and how we can use simulations to improve [9:41]
      • Making decisions in groups and the importance of diversity [16:35]
      • Thinking thousands of years ahead [22:23]
      • How ideas come from niche groups, and a discussion of managing the chaos of the internet [27:21]
      • Practical advice for long-term planning [36:15]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and I’m here today with Chris Dixon, a general partner on a16z Crypto, and Steven B. Johnson, who is the author of many books, including “Where Good Ideas Come from,” the PBS series, “How We Got to Now,” a book on play called “Wonderland,” and his latest book is “Farsighted,” which is, “How We Make the Decisions that Matter the Most.” So, welcome.

      Steven: Thank you for having me.

      Long-term decision-making

      Chris: Could you start just telling us a little bit about the book?

      Steven: Yeah. This is a book that has been a long time in the making, which is appropriate for a book about long-term decision-making. It had a long incubation period. One of the things that occurred to me, that got me interested in this topic, is that there had been a lot of material written, both in terms of academic studies but also in terms of kind of popular books — but a disproportionate amount of that was focused on people making gut decisions or instinctual decisions.

      Chris: Just like thinking fast.

      Steven: Thinking fast and slow. Also “Blink” is like that. It is amazing the amount of processing and all the heuristics we have for making short-term instinctual decisions. But the decisions that really matter the most are slow decisions, or decisions that have a much longer — both time span in terms of how much time you’d spend deliberating them, and then also the time span of their consequences. And I got interested in what, kind of, the science is, and some of the art in a way, behind those kinds of decisions. Actually, the book partially starts with a great excerpt in Charles Darwin’s diaries where he’s trying to decide whether to get married. And it’s a beautiful list where he’s like, “Okay, against getting married, I’ll give up the clever conversation with men in clubs.”

      Sonal: My favorite of “against marriage” was “less money for books, etc.”

      Steven: Yeah, yeah. Right, right. And it’s this list, and, you know, looking at it, it’s kind of comical and sweet in some ways, but that technique of creating a pros and cons list, basically, that was state of the art in 1837, 1838, and it’s still kind of state of the art for most people. That’s the one tool they have for making a complicated decision. And, actually, we have a lot more tools, and we have a lot more insight about how to make these things.

      Chris: It seems like there’s two questions, right? There’s a descriptive and a normative question. Like, how do people make these decisions, or societies, or, you know, governments, or whoever the actor might be? And then there’s a second question, how one should make these decisions, right?

      Steven: I got more and more interested in the second question, right? Like, what are the tools that you can really use to do this in your life?

      Chris: And can you get better at it?

      Steven: Yeah, and it’s a tricky one. I was really grappling, trying to take very seriously the legitimate objection to a book like this, which is that it is in the nature of complex life decisions, career decisions, “should I get married” decisions, “should I take this job” decisions — that each one is unique, right? That’s what makes them hard, is that they’re made up of all these multiple variables and competing value systems and stuff like that. And it turns out, really, that a lot of the science of this and the, kind of, practice of making a deliberative decision, is a set of tricks to get your mind to see the problem, or the crossroads, or whatever you want to call it, in all of its complexity, and to not just reduce it down to a series of predictable patterns or clichés or stereotypes. And that’s where actually the advice, I think, is useful.

      Chris: And so that’s, like, the scenario planning where there’s, sort of, a discipline around — what’s the upside case, the middlecase, the frameworks, forcing yourself to, kind of, mentally traverse different future paths.

      Steven: Yeah, exactly. Well, one of the big themes of the book that runs throughout it in lots of different ways is the importance of storytelling.

      Sonal: Like a narrative.

      Steven: Yeah, and all these different ways. Scenario planning is one example, and that’s usually used in a, kind of, business context, right? So you’re like, okay, we’re trying to decide should we launch this new product. Let’s generate some scenario plans for what the market is going to do over the next five years, but let’s generate multiple ones. Let’s not just predict the future.

      Chris: Yeah, I had a friend once who worked at a large oil company in their scenario planning group. And, you know, at first, it doesn’t sound, like, that interesting. But it turns out these large oil companies, like, whether oil is $30 or $100, you know, a lot of money is at stake. And so they had this infrastructure, like, thousands of people. It was, like, the state department or something. It was quite fascinating to hear about, like, what if there’s a war in this area and oil drops this much, and what do we do and, like, just the level of rigor. I never imagined it was as complex as — you know, as sophisticated as it was.

      Steven: Well, I had some great conversations over the years with Peter Schwartz who’s here in the Bay Area, and he’s one of the pioneers of scenario planning. And one model that he talks about is you do three different narratives — one where things get better, one where things get worse, and one where things get weird.

      Sonal: That’s interesting. I’ve never heard that.

      Steven: Yeah, I love that, because I think that all of us kind of intuitively build the, like, “it gets better, it gets worse” kind of scenario plan in our head. It’s useful to actually walk through it, and do it, and tell that story. But the weird one is what’s cool, because then you’re like, “What would be the really surprising thing?”

      Chris: Well, the funny thing, at least, if you look at history, weird is often the case.

      Steven: That’s right. We’re living through it right now, and that’s for sure. And a key part of it is that the predictions don’t even have to be right on some level for it to be useful exercise, because a lot of this is about recognizing the uncertainty that’s involved in any of these kinds of choices. It’s creating a mindset that’s open to unpredictable events. So, going through narratives where you imagine alternatives — even if they don’t actually turn out to be the case, they get you in a state so that when you do encounter an unpredictable future, whatever it happens to be, you’re more prepared for it, or you’ve thought about at least some of those variables.

      But the other thing I was just going to say on the storytelling front, one of the places where it, kind of, came together — there’s a lot in the book about collective decisions. Like, what do we do about climate change, or what do we do about the potential threat from superintelligence and AI, right? Something that we think about a lot here.

      Sonal: Global, multi-generational type of things.

      Steven: Yeah, super long-term decision-making, right? And one of the points that I tried to make in the book is, while we have this cliche about our society — that we live in this short attention span world and we can’t think beyond 140 characters and all that stuff — the fact that we are actively making decisions that involve changes to the environment that might not happen for another 20 or 30 years, and we’re thinking about what the planet might look like in 100 years, is something that people have not really done before. They’ve built institutions designed to last for longer periods, so they’ve built pyramids designed to last, but they weren’t very good at thinking about, you know, “We’re doing these things now. What will be the consequences 80 years from now from these choices we’re making now?”

      Chris: So, regardless of what you think about whether we’re doing enough for climate change now, the very fact that it’s a central political topic — that was not the case 100 years ago.

      Steven: It’s a sign of progress. And superintelligence is even a better example of it, I think, because the fact that we’re having a debate about a problem that is not at all a problem for us now, but then potentially might be a problem in 50 years — that is a skill that human beings didn’t used to have. When I was talking about this once with Kevin Kelly out here, another Bay Area person —he had this great point which is, like, this is why science fiction is such an important, kind of, cognitive tool, because you run these alternate scenarios of the future and they help us, kind of, imagine what direction we should be steering in, even if they’re made-up stories.

      Sonal: Don’t people actually say that science fiction is the only way to “predict the future” in terms of what you can actually think of for very complex technologies? I feel like I’ve heard a statistic or an observation to that effect.

      Steven: I mean, I’d certainly think that you would find more things that ended up happening in fictional accounts than, you know, official people making predictions about the future outside of a fictional context.

      Chris: Yeah, my bias has always been towards history, for example. Like, the only way you’re ever going to possibly get a lens on how to predict the future is to read a lot of history, understand how these things work, because of social complex systems. You’re not going to, you know, have empirical data, and polling, and everything else to analyze this stuff. I wonder to what extent our ways of thinking about these things in academic literature and things like this have been shaped by the kind of the — you know, when you require everything be testable, you also dramatically narrow…

      Steven: The things that can be tested.

      Chris: Yeah.

      Steven: Or, the things that can be tested [are] a subset of the things that are interesting and worth exploring in the world. And you get steered towards those things. I made this decision with my wife to move to Northern California, having lived in Brooklyn and New York for a long time. And, you know, when you think about a choice like that, there are so many different variables. There are variables about the economics of it, the kids’ schools, do you want to live in a city or do you want to live near nature. I mean, all these different things, it’s an incredibly complicated thing to do…

      Chris: All the second-order things you could never predict.

      Steven: Right, what will the consequences of it be?

      Chris: The serendipitous meeting your kid has, the changes in your life, or…

      Steven: Yeah, particularly with children, you know you’re changing the overall arc of your kid’s life by making a choice like that, and that’s scary. But to your point, that kind of decision — well, certainly I would say is one of the most important decisions that I ever really thought about and kind of worked through with my wife. How would you study that in the lab, right? You know, it’s very hard to, like, be like, “Okay, everybody, we’ve got 10 of you that are going to move, and there’s another 10 of you that…” And there’s no, like, double-blind study you could do.

      Using simulations to make decisions

      Chris: And by the way, that’s why — you mentioned in the book simulations, and we have actually some investments in this area, but, like, the idea that computing is getting powerful enough that you could ask questions like, “We want to fix the New York subways, and we want to shut down these subways. How does that have — what are all the consequences of that?” Or, we change interest — you know, there’s always been the Santa Fe kind of…

      Sonal: The complexity.

      Chris: …you know, the complexity theory simulation. I think it’s still kind of this fringe. I always think about — I have friends who did machine learning in the ’80s, and back then it was this kind of rebel fringe group in AI, right? So mainstream AI back then was heuristics-based. It’s like, okay, we’re going to win all these things by, you know, literally putting in these rules and teaching computers common sense. And there was this, kind of, rebel group that said, “That will never work. You need to use statistical methods and have the machine learn.” Now, fast forward to today, like, machine learning and AI are synonymous, right? It feels like simulations today are this, kind of, fringe group.

      Over time, like, it just seems, like, a far better way to test these really complex things. Like, what if you could run a simulation — I don’t know if you could run a simulation for moving to California, but you could run a simulation for changing interest rates or for closing down a bridge. Those things, I think, are fairly limited today. You could imagine them getting orders of magnitude more sophisticated, right?

      Steven: There’s so many things to say to that. So the first is, it actually gets back to that classic book that David Gelernter wrote in the ’70s or ’80s.

      Sonal: Oh, my God, “Mirror Worlds.”

      Steven: “Mirror Worlds” and that was a…

      Sonal: I edited him on a theme post after that. He’s one of my dear favorite people.

      Steven: I read that book when I was, I guess, just in grad school. It was one of the first tech books where I was like, “Oh, this is really fascinating.” In some ways, my first book was shaped by that.

      Sonal: Marc Andreessen also said it had a huge influence on him.

      Steven: Yeah, yeah. And so, we will — I think that is something that’s coming.

      Chris: We should explain “Mirror Worlds.” The idea is that, as I recall, you kind of have the whole world instrumented with IT devices and things. And then the Mirror World is the computer representation of that, and the two can interact in really interesting ways.

      Steven: Yeah, so basically you have every single object in the — and let’s say we’re talking about a city, you know — is somehow reporting data on all of its different states. And then the computer is just some massive supercomputer, although it was a supercomputer in his day. Now it might just be like an iPhone or something.

      Sonal: He, by the way, today argues it’s just streams of information.

      Steven: Right. Yeah, yeah. What was that thing? It was like lifestreams or something.

      Sonal: He had a lifestreaming thing, but now he thinks about it in the context of streams, as like browsers, Twitter, like, streams of information that we constantly live in.

      Steven: So you basically have, you know, software that’s looking in all that information, and then the idea would be that it would develop enough of kind of an intelligence that you could say, “Given the patterns you’ve seen over the last 10 years with all these different data points, if we close that bridge, or if we, you know, switch this one neighborhood over to commercial development, what would it look like? Press fast-forward. It becomes a kind of SimCity kind of simulation but based on actual data that’s coming from the real city. It’s just one of those ideas. I think there’s a whole generation of books you’ve, kind of, read.

      Sonal: Yeah. I always think of “Ender’s Game” and the whole scene where he essentially is playing a simulation and he realizes in the end — I mean, I’m sure this book’s been out for years.

      Steven: Spoiler alert.

      Chris: Hey, don’t spoil “Ender’s Game.”

      Sonal: But that it’s actually the real war that he’s fighting in the final simulation.

      Steven: So, the other thing about simulations — it is a big theme of the book. It’s one of those, kind of, ways in which the book connects to storytelling as well, because I think the personal version of this for the “should I marry this person or should I move to California” — this is actually what novels do, right? We don’t have the luxury of simulating an alternate version of our lives, because we can’t do that yet. We probably won’t be able to do that for a long time, particularly the kind of emotional complexity of choosing to marry someone or something like that. But we do spend an inordinate amount of time reading fictional narratives of other people’s lives. And the idea is that that’s part of the — almost, like, evolutionary role of narrative is to run these parallel simulations of other people’s lives.

      Sonal: That’s a fascinating way of putting it.

      Steven: Right? And by having that practice of seeing, “Oh, it played out this way with this person’s life, this way with this other person’s life.” And the novel’s ability to take you into this psychological…

      Sonal: Immersive.

      Steven: …of what’s going on in a person’s mind. A great biography will do that, too. So reading history, as you said, is a part of that. But it’s — in fact, the first draft of this book had just, like, a ridiculous amount of “Middlemarch” in it.

      Sonal: You still have a lot of “Middlemarch” in it, for the record.

      Steven: It was right up front in the first draft, and I think my editor was like, “This is great, but I don’t know if this is what people need.” It’s interesting how we spend so much time either, kind of, daydreaming about future events, or reading fiction, or watching fiction on TV. We spend so much time immersed in things that are, by definition, not true. They haven’t happened or they haven’t happened yet. And I think the reason we do that is because there’s an incredible adaptive value in running those simulations in our heads, because then it prepares us for the real world.

      Chris: We’re building, kind of, the emotional, logic space or something in terms of, I don’t know, expanding. I always think of that — like, I always get this feeling when I read a good book. I think someone said it makes the world feel larger, right, and I think it’s another way of saying it, kind of, expands, you know, the possible, like, trees of possibility, right?

      Sonal: It’s like your mental sample space.

      Chris: Yeah, you just feel like the world is bigger, right? You read history and you feel like it’s big — or you read a novel and you feel like the emotional world is bigger, right, and there’s, sort of, more possibilities. And it’s interesting, so you’re saying it’s almost like an evolutionary need to do that to adapt, to be more emotionally sophisticated.

      Steven: There’s a great essay by Tooby and Cosmides, I believe the names are pronounced, about the, kind of, evolutionary function of storytelling. And they — one of the things that they talk about is precisely this point, that we spend an inordinate amount of time thinking about things that are not true, and that would seem to be actually a waste of time. But in fact, there’s a whole range of different ways in which things are not true. There’s the, “She said it was true, but it’s not true,” or, like, “This might happen and thus might be true, but it’s not true now.” Or, you know, “I wish this were true.” And our brain is incredibly good at bouncing back and forth between all of those, kind of, hypotheticals and half-truths. And I don’t mean this in a kind of “fake news” kind of way. Like, this is actually a really good skill — the ability to conjure up things that have not happened yet but that might is one of the things that human beings do better than any other species on the planet as far as I know.

      Sonal: It allows us to create the future.

      Chris: Like, and also to do it a — I think Aristotle said the point of tragedy was that you could experience it with an emotional distance, right? So, you can go — that’s another value of narrative, right — you can go and you can experience and, like, look at the logic without — so you can go and think about tragedy and how to deal with it without actually being overwhelmed by the emotion of it, right? And so you’re involved but not so involved that you can’t, sort of, parse it and understand it, right?

      Making decisions in groups

      Steven: That’s a great point. And the other thing, I would — just a last point on simulations. We’re talking about how it’s hard to simulate these types of decisions in the lab, but the one place in which we actually have seen a lot of good research into how to successfully make complex deliberative decisions is another kind of simulation, which is mock trials and jury decisions, right? And that gets you into group decisions, which of course is a really important thing, particularly in the business world.

      Chris: So, like, what are the key, I guess, components both to the group composition, and also to the process to determine, you know, to get to the right answer?

      Steven: So the biggest one, which is something that’s true of innovation as well — not just decision-making — is, you know, diversity. It’s the classic slogan of, like, diversity trumps ability, which is — you take groups of high-IQ individuals who are all from the same, say, academic background, or economic background and have them make a complicated group decision. And then you take your group of actually lower-IQ people, but who come from diverse fields, professions, fields of expertise or economic fields, whatever, cultural background — that group will outperform the allegedly smarter group.

      Chris: Is that because that more diverse group will traverse more future paths of the tree of possibilities?

      Steven: So, the assumption was always — the diverse group just brings more perspectives to the table, right? So, they have different — you know, it’s a complicated, multi-variable problem…

      Chris: That’s going to your earlier framework. Is that good, bad, weird? Like, they’ll just simply bring up and explore more possibilities, because of their more diverse experiences?

      Steven: There’s no doubt that that’s part of it, right? What makes a complex decision complex is that it has multiple variables, operating on, kind of, different scales or different — you know, and it’s a convergence of different things.

      Sonal: Right, you’re saying it’s more nuanced than that.

      Steven: So, it also turns out that just the presence of difference in a group makes the, kind of initial, kind of, insiders more open to new ideas. If you have, kind of, an insider group, a homogeneous group, and you bring in folks who bring some kind of difference — even if they don’t say anything — the insider group gets more, kind of, original.

      Sonal: They rise to the occasion.

      Steven: They challenge their assumptions internally more. So, there are exercises you can do to bring out the, kind of, hidden knowledge that the diverse group has — the technical term for it is hidden profiles. And so when you put a bunch of people together and they’re trying to solve a problem, come up with a decision, there’s a body of, kind of, shared knowledge that the group has. This is the pool of things that everybody knows about this decision that’s obvious.

      For the group to be effective, you got to get the hidden pieces of information that only one member knows, but that adds to the puzzle, right? And for some reason, psychologically, when you put groups together, they tend to just talk about the shared stuff. Like, there’s a human kind of desire to be like, “Well, we all agree on this.” And so some of the exercises and practices that people talk about are trying to expose that hidden information, and one of them is just to assign people roles and say, “You are the expert on this. You’re the expert on this. You’re the expert on this.”

      Chris: Just arbitrarily. So they say, “My job is to go and be the expert on this, and therefore I’ll more likely surface hidden knowledge.”

      Steven: Yeah, it diversifies the actual information that’s shared, not just, like, the profile of people.

      Sonal: I have a question about this, because I find that fascinating, that you can essentially define expertise as a way to go against the problem of seeking common ground. But then later, you talk about this difference between the classic phrase of foxes and hedgehogs, and how actually it’s not hedgehogs that are deep experts in a single thing, that perform well in those scenarios, but foxes that are more diverse in their expertise. So I couldn’t reconcile those two pieces of information.

      Steven: That’s a great question. So, just to clarify — so it comes out of this famous study that Philip Tetlock did.

      Sonal: He wrote “Superforecasting.”

      Steven: Yeah, yeah, and “Expert Political Judgment.” And he did one of the most amazing, kind of, long-term studies of people making predictions about things. And it turned out, kind of famously, that all the experts are, like, worse than a dart-throwing champ at predicting the future. And the more famous you got, the worse you were at predicting. But he did find a subset of people who were pretty good, you know, significantly better than average of predicting kind of long-term events — which of course is incredibly important for making decisions because you’re thinking about what’s going to happen. You can’t make the choice if you don’t have a forecast of some kind. And what he found in those people — he described them in the classic fox versus hedgehog which is, you know, the hedgehog knows one big thing, has one big ideology, one big explanation for the world. The fox knows many little things, and is a kind of monolithic thinker but has lots of, kind of, distributed knowledge.

      And so the reason why that, I think, is in sync with what we’re talking about before is, in that situation, you’re talking about individuals. So, it’s a fox and a hedgehog. And what the fox does is simulate a diverse group, right? He or she has a lot of different eclectic interests. And so inside his or her head…

      Sonal: Right. There are, like, 10 people in their head.

      Steven: Right. That’s one of the reasons why, you know, a lot of the people who really are able to have these big breakthrough ideas — one of their defining characteristics is that they have a lot of hobbies.

      Sonal: Oh, that’s so true. I used to give the tours at Xerox PARC for all the visitors, and actually one of the big talking points was, when we had, like, one of these big muckety-mucks coming through — was, like, how there’d be a material science expert, and he’d be the world’s expert in, like, goat raising — or there’d be someone else who’s a father of information theory for computers, and he’s, like, a world-class surfer. They all had one specific, like, music, whatever.

      Steven: Yeah, there’s a funny connection actually to “Wonderland,” my last book, which is all about the importance of play and driving innovation. And so much of, kind, of hobby work is people at play.

      Sonal: Right, Dixon has a classic post on this, on, like — the things that the smartest people do on the weekend is what the rest of the world will be doing 10 years later.

      Steven: Yeah, I remember reading that.

      Extremely long-term thinking

      Chris: Yeah, I mean, the way I was thinking about [it] is, there’s so many things in life — especially the workplace — are governed over — you basically have a one to two-year horizon, right? And that’s particularly because business people almost by definition, right, if you work in a public company, they’re moving by quarter, by year. And so where are the places in the world where smart people have a ten-year-plus horizon? I mean, it’s, like, probably academia? And then my model would be sort of technical people on the weekends — nights and weekends, right?

      I think it’s more than a coincidence that so many of these, you know, Wozniak and Jobs, and the early internet and all these other things started off as these, like, home-brewed clubs and weekend clubs and things like that, right? Because it’s just simply time horizon, right? I mean, I think it relates to your book but, like, so much of what we’ve done or what we do in the business world, and just the whole, kind of, system, right, is structured around a relatively short time horizon. I think about it in terms of, like, what we do in our job. One of our big advantages is the fact that we are able to take a longer-term perspective, just based on where capital comes from and all the other kinds of things. And that just le’s you invest in a whole bunch of things that other people just simply can’t because they’re under a different set of incentives.

      Steven: I mean, one of the great things that I got out of actually deciding to move to California is spending a bunch of time with the folks at the Long Now Foundation. You know, it’s really trying to encourage — it’s not 10 years. It’s, you know, 1,000 years.

      Sonal: 10,000. It’s a 10,000-year clock, literally.

      Steven: Basically, it would be as long — to last as long in the future as civilization is old. Yeah, I tell people about that. They’re like, “That’s an incredibly idiotic waste of time. Why would you want to <inaudible>? There’s so many pressing problems.” But so many of the problems we have now come from not having taken that kind of time, right? And, in fact, one of the other riffs in the book — I started thinking about like, “Okay, if we are now capable of thinking on longer time scales — if we’re thinking about climate change on 100-year scale, if we’re thinking about superintelligence on a 50- or 100-year scale, what’s the longest decision that one could contemplate?” And actually, Zander Rose who…

      Sonal: He runs The Interval for The Long Now.

      Chris: …runs The Interval at Long Now. He heard me talking about this, and he said, “Oh, we’re working on this project with this group called METI, which is a group that is debating whether to and what they should — if they decide to — send as a targeted message to planets that are likely to support life.” Now, we’ve identified these planets, whatever. And it’s similar to superintelligence, in that it’s a surprisingly controversial project, and there are a bunch of people, including the late Stephen Hawking, who thinks it’s a terrible idea.

      Sonal: And if you’ve read “The Three-Body Problem,” it’s the worst idea ever.

      Steven: Exactly, yeah. “The Three-Body Problem.” I’m sure a lot of your listeners have read that.

      Chris: It just provokes them.

      Steven: By definition, they are going to be more advanced than we are, which is a whole complicated reason why that is, but they will be. And in the course of human history, every encounter between a more advanced civilization and a less advanced civilization has…

      Sonal: Ended in a bad way.

      Steven: …ended badly.

      Sonal: And this is, by the way, rooted in the Drake equation and the Dark Forest analogy.

      Chris: Yeah, and the Dark Forest idea, right, is that therefore the best strategy is to be…

      Sonal: To be silent.

      Steven: Yeah, that’s right.

      Chris: We should keep it on the down-low.

      Sonal: You hunt silently, or you don’t hunt.

      Chris: And that’s the answer to the — was it Fermi paradox?

      Sonal: Right, Fermi paradox. Exactly, it brings all these concepts together.

      Steven: What I just love about it is just, because of the speed of light and the distance you have to travel to these planets, this is a decision that, by definition, can’t have a consequence for at least, you know, 5,000 to 50,000 years, and depending on the planet you’re targeting, maybe 100,000 years. And so the idea that humans are walking around and be like, “All right, I think we’re going to decide to communicate with these aliens on this other planet, and we’ll get the results back in 100,000 years.” Just the fact that we’re capable of thinking that is pretty amazing.

      Sonal: You know, I find something kind of, not self-indulgent, but something that, I think, is very confusing about making decisions in this framework, is that — you know, we can’t predict 10,000 years ahead, but nor can we predict immediate second and third-order effects of things we build today. So my question is — I mean, this sounds like a terrible question to ask, the book is about making better decisions — but why bother making a good decision? Why don’t we just, sort of, let it work itself out in a series of complex, little, tiny events?

      Chris: You’re saying why bother because you can’t do anything…

      Sonal: You can’t predict the future. I mean, we don’t how things are going to play out.

      Chris: Yeah, well, the question is, can you get better at it? I think that was the thing — I think that’s one of the things that’s important about Tetlock’s work which is — that first book was about people being comically bad at it, but he did carve out this element and said some people actually have a strategy that works and seems to be better than just flipping a coin or, you know, you just making it up. And so, I think that, you know, there’s definitely not a crystal ball for this, and there’s not an applied strategy that works in all situations, but I do think you can kind of nudge it. And because decisions are — I mean, that is, kind of, the definition of wisdom, is that you make the right choices…

      Sonal: Right, you make a decision.

      Steven: …in life, right?

      Ideas from the fringes

      Sonal: I have a question, too. So we talked a little bit about the fox and the hedgehog. One of the things you mentioned in your book is the role of extreme perspectives versus mainstream, and I thought that’d be really interesting because we think about that a lot. Like, where ideas come from on the fringes.

      Steven: Well, it all kind of revolves in the story about the Highline in New York, right? The now-iconic park that was an old, abandoned rail line. One of the…

      Sonal: On the West Side Highway.

      Steven: Yeah, one of the great urban parks created in the 21st Century. And for, you know, 20 years, it was an abandoned rail line, an eyesore, a public nuisance, and so on. So, one thing that the book argues is, there’s a stage in decision-making, in the early stage, which one should consciously, kind of, seek out to do — which is to diversify your options, right? And folks have looked at — one of the key predictors of a failed decision is, it was a “whether or not” decision. There was just one alternative, like — should we do this or not?

      Sonal: In a company.

      Steven: In a company, but I think it applies to a lot of things. When you just have one option on the table, those decisions are more likely to end up in a, kind of, failure of one form or another. So part of the strategy, as I said, when you’re at that early stage — let’s do this versus this versus this. Multiply your options. In the case of the Highline, for 20 years, the debate about the Highline was basically, should we tear it down or not? And it was really even agreed that we should tear it down, but just who’s going to pay for it? It was like, it’s a rail line that nobody is using. Industrial rail is not coming back to downtown Manhattan, whatever. And so it was just stuck in this kind of “whether or not” form. And then this interesting bunch of folks, who, to your kind of point about extreme positions — who were not part of the official decision-making process of what to do — that was the city. It was a debate between the rail lines and, you know…

      Sonal: <inaudible>

      Steven: But then you had, you know, an artist, and a photographer, and a writer who’d kind of gotten attached to this idea that maybe you could do something with this space. And it was this, kind of, marginal set of folks, who were not part of the official conversation about what to do with this, who added a second option — you know, and said, “Listen, what if we kept it and turned it into a park? That would be amazing.” Because our politics are so contentious and polarized, there’s this, kind of, default — you know, anti-extremism now. Like, we want to get out — you know, we get rid of this extremism. But in a society, there’s a certain level of extremism that’s really important. So, sometimes ideas that are important and need to happen come into the mainstream from the margins. So, it’s trying to get, what I call, the optimal extremism. And it’s a tricky one. I don’t have, actually, a clear recipe for this, but, I think — when you’re making a decision, are you bringing in those fringe voices to at least have a seat at the table?

      Chris: Relating to the internet, like, one thing I think is so potentially great about the internet is you have all of these niche communities. You know, subreddits and, you know, crowdfunding. You know, we’re investors in Oculus, and I don’t think Oculus would have ever gotten initially funded had it not been for the crowdfunding. I mean, there’s obviously been, you know, bad things on the internet as well, but I think, for the most part, I believe [it] has allowed some of these kinds of more interesting and potentially positive fringe groups to get together. Whether that will continue, you know, as the internet has become more and more centralized, is a topic that we both have talked about before. You wrote a really interesting article for the New York Times last year about something I spent a lot of time on.

      Steven: It was a kind of adaptation of your work actually.

      Sonal: Oh, that’s awesome. That’s actually great to hear.

      Chris: A much better version of it. But, yeah, so, you know, I think the issue we were talking about is sort of the centralization of the internet, and how do we make sure that the internet stays interesting and diverse and, I think, good for small businesses and creators, and all sorts of other people, right? And this is an issue that I think a bunch of people are talking about, right? I mean, you see it discussed when people talk about these issues like demonetization, deplatforming. You see people talk about it in terms of regulation, should these platforms be more regulated? Are we headed to an internet that’s similar to TV, where you have, like, four channels that control everything. You know, Google, Facebook, Amazon, etc. And then you wrote about — there’s this, kind of, fringe movement that is trying to, kind of, through technology principles and innovations, create alternative infrastructure.

      Steven: Yeah, there was a direct connection, actually, between “Farsighted,” this book, and that piece for the Times Magazine. And really the thing that began it all was Walter Isaacson wrote an op-ed, I think, in “The Atlantic,” saying the internet is broken, you know, and we need to fix it. It has these problems. And he kind of listed a bunch of problems, which I thought were reasonable. And so, I sent him a note, and I said, “You know, I liked what you wrote. How would we go about fixing it? Like, what would be the decision-making body that would decide these are the fixes and we’re going to apply them?” And he wrote back and he said, “You’re right. It would be impossible in this polarized age. You know, we can’t do it.” And I thought, “That’s incredibly depressing, right?”

      Sonal: That’s not a good answer.

      Steven: Like, you know, if we’re just stuck with the infrastructure we have, then that’s really depressing, right? So I slowly, kind of, dug through the writing about it, and, you know, about halfway through it, I began to think that some of the blockchain models and some of the token economy stuff that you’ve written about as a way of creating sustainable business models for open protocols, basically — which is what we really, kind of, need. I think one of the reasons that piece worked is that — there were a million pieces written about the blockchain, but I didn’t actually set out to write a piece about the blockchain. I set out to write a piece about how would we fix this problem, and I got organically led towards the blockchain. Meanwhile, as that was happening, all the crazy ICO scams were happening, and, like, it was like the best and the worst of online culture exploding all around me.

      Chris: I think I read the same thing. Walter Isaacson, I think he articulates very well the negative side of it. I think the positive side — I would argue two things. Like, one is just the nature —  like, the architecture, specifically the internet protocol, being very presciently designed as a dumb layer in a good way, right, so that you can reinvent. The internet is reinvented if the nodes on the internet upgrade themselves, right? And so I think of internet architecture as the intersection of incentives and technology design, right? So you have to create a better kind of software that runs in those nodes, and then you have to provide the right incentives, right? And one of the fascinating things about the bitcoin whitepaper is it’s, essentially, you know, eight pages of incentives. And if you do the incentives right, the internet is able to, sort of, heal itself or upgrade itself, I should say, or change itself. And then the question people are looking at is, can you take that interesting incentives design, and can you apply it for things that are more useful than simply solving cryptographic puzzles like bitcoin, right, and incentivize new behavior?

      So the other thing I always think about is, so many of the models we use are hardware-based, including — I’ve read all your books and, like, the people you talk about, right, by definition, are building usually physical things, because that’s what they were doing 20 years ago, right? And you think about, like, once you build the combustible engine, you basically built it. I mean, you can improve it. You know, you build a car, you basically build it. Where software is fundamentally different. This is a Marc Andreessen point — “software eats the world.” He’s always talking — he just thinks people fundamentally misunderstand software, and keep applying these old physical models of how — you know, Carlota Perez, and all these — which are great frameworks but they’re all based on how hardware cycles work, right?

      Steven: Yeah. I guess one thing that I would, kind of, bring out that I actually didn’t get to in that crypto piece in the Times was the importance of governance — structures inside of these crypto protocols and platforms. And, you know, there’s always been some level of governance involved in software, in the sense that you had a corporation, or you had a standards body that was, you know, deciding what the actual software package should be, or what features should be included. But now, really, for the first time, the governance is actually built into the code. If you think about decision-making that — that is, in a sense, you know, do you have governance? Like, we have embedded in this code a set of rules, governing, like, what we collectively are going to decide for the future of this platform. And the fact that that’s now being built into the software is really fascinating.

      Chris: Well, the point of this movement is to decentralize, take the power away from an individual, and therefore you have to think about, well, then how do these systems upgrade themselves and govern themselves? And who gets to decide who gets a voice? And all these questions, right? Because in the old model, you just said, okay, the CEO. Right now, it’s like, “Well, there’s no CEO,” so how do you figure it out?

      Advice for long-term planning

      Sonal: For masses of people to decide and coordinate activity at an unprecedented scale. So this has been great. We’ve been talking about decision-making and how it plays out, you know, in crypto, in innovation, and also then even in personal lives — like Darwin, or even novels and literature like “Middlemarch.” But what are some concrete takeaways or advice — not just for how to think about decision-making and being farsighted, but for what both people and companies, big or small, could do?

      Steven: So, for instance, one of my favorite kind of tricks in the book is this thing that Gary Klein came up with, which is a technique also to deal with, kind of, the dangers of groupthink in making, let’s say, a work decision, where you’ve got your team and you’ve decided, “We are going to launch this product and we’re all really excited about it.” And so, he created this, kind of, technique which he calls a premortem. I love this idea. So postmortem, obviously, the patient is dead. You’re trying to figure out what caused the patient’s death. A premortem is — this idea is going to die a spectacularly horrible death in the future. Tell the story of how that death happened, right? In five years, this will turn out to have been a bad decision. Tell us why. And that exercise — again, it’s like scenario planning. It’s a kind of negative scenario planning. Even if it ends up not being true, the exercise of forcing your brain to come up with a story…

      Sonal: The alternative thinking.

      Steven: …as opposed to just saying, “Hey, guys, do you see any flaws with this plan?”

      Sonal: Do you guys do that when you talk through deals?

      Chris: Yeah. No, so I think a good investor discipline is to do something similar to that where you kind of — and frankly an entrepreneur — I think one of the myths around entrepreneurship is that there — I mean, they’re risk-takers. That said, entrepreneurs do take risks, but good entrepreneurs are very good at doing premortems, ordering the risks, and then systematically trying to mitigate them, right? I mean, now, that’s not to say that they don’t take big risks, but you certainly don’t want to take unnecessary risks, right? So I think what a good entrepreneur is doing is constantly thinking about all the different scenarios, how they’ll go wrong, you know, kind of, rank-ordering them, taking a bunch of risks, but saying — hey, so my key risk and this is — you know, it’s sort of like, “This type of business is all going to be financing risk, and this one will all be about talent, and this one will be all about, you know — how will it go wrong.” And you see enough of it — and, of course, it’s a very rough and imperfect science, but it feels like you can get — it seems like you get better over time.

      Steven: Yeah, the original patent that Google filed for the self-driving car projects — included in it is this thing they call the bad events table. Basically, it’s like, at any moment as the car is driving, it’s creating this bad events table, and the bad events are ranged from, “I’m going to dent the right side mirror by accident, you know, just scraping against this car,” to, “I’m going to collide with these two pedestrians and they’re going to die.” And there’s, like, 15 bad events that can potentially happen, given the circumstance in the road. And not only do they, kind of, list the bad events, but then the software is calculating both [the] likelihood of the event happening, and then the magnitude of the risk, right? So two pedestrians die — very high magnitude — but if it’s very low probability, you kind of measure it. And I think of that as, in a sense, the car is doing that at the speed of instinct, but in a way, that’s a kind of table that would be really nice to put next to a pros and cons table, you know? What are all the terrible things that could happen? And let’s rank them with probability and with magnitude. Just to see it.

      Sonal: I think about this all the time, actually, in terms of how people make pros and cons lists, and how they’re so flat variable-wise. And if you’ve gone through any statistical training, the first thing you learn in any linear model is how to weight your algorithm, and you weight the variables. And I always think about that. Like, well, I’m going to give this, like — well, I’m going to give this, like — a move to California 10x weight, and my move back from New York. You’ll give something else 2x, and you multiple all those probabilities and those weights to come up with your decision. I think that’s a very good way of thinking about it.

      Steven: You know, pros and cons tables date back to this famous letter that Ben Franklin writes to Joseph Priestley — who, coincidentally, was the hero of my book, “The Invention of Air” — but he’s, like, explaining this technique he has, which is basically a pros and cons list, and he calls it moral algebra. What gets lost in the conventional way that people do pros and cons lists is, Franklin had a kind of weighting mechanism, where he basically said, “Okay, create your list of pros and cons, and then if you find ones that are comparable, kind of, magnitude on one side and the other, cross them out.” We would do it differently now, but it was a way of assessing, “Okay, these two things are kind of minor, and I got one on one side, one on the other, so I’m going to cancel that out.”

      Sonal: They don’t make — that’s great.

      Steven: I think some of those exercises are really important. I think cultivating a wide range of interests and influences is a really important thing to do, both in terms of innovation and creativity, but also in terms of decision-making. And I think it’s very important to stop and say, “Okay, what would the alternate scenarios be? What if it gets better? What if it gets worse? What if it gets weird?” And the other thing about the, kind of, diversity point I think that’s going to become increasingly important — the diversity is actually going to be also machine intelligence, too, right? Increasingly, part of that intellectual cognitive diversity is going to involve machine intelligence.

      Sonal: Oh, interesting.

      Steven: And so it’s going to be, you know, not just, you know, making sure you have a physicist and a poet in your, kind of, posse that’s helping you make this decision — but we’re going to see more and more people making decisions. For instance, you know, there’s a lot of interesting research, in the legal word — bail decisions. Normally, a judge would make a decision, “Okay, this person should be let out on bail for this amount, or not let out on bail, whatever.” And there’s some evidence now that machine learning can actually make those decisions more effectively. It’s not that we want to hand over the process to the machines entirely, but the idea that you would be assisted in making a choice like that, I think, is going to be something we’ll see more and more of.

      Sonal: I mean, I think we’re already seeing hybrids of that play out, like, with hedge funds with quant strategies, etc. But you’re saying something even more. You’re saying it’s like a partner in decision-making.

      Steven: Yeah, it’s a collaborative model. My friend Ken Goldberg, who’s at Berkeley in the robotics program there. He talks about inclusive intelligence, right? The idea that it’s not just about, you know, just human intelligence versus artificial intelligence, but actually this, kind of, dialog that you’re going to have with a machine. You might say, “I think I should release this person on a very low bail,” and the machine comes back with, “Well, looking at all comparable case studies, I think he actually, you know, shouldn’t be released at all.” At that point, you’re like, “Okay, that’s interesting. I’m going to question my assumptions here and think about what I might have missed.” You might not change your mind, but having that extra voice in the long run will probably be better for us.

      Sonal: Right. It feels like crowd intelligence on a whole massive different scale. Were there any qualities of people that you’ve seen? One of the things that you put in the book was that one of the key factors is an openness to experience as a real great predictor — a very good decision-making prediction, etc. I thought that was fascinating, because I thought of immigrants. It’s like a defining quality of immigration, and what brings people to different places.

      Steven: You know, it’s one of the big five personality traits.

      Sonal: It’s openness to experience.

      Steven: It’s another…

      Sonal: It’s another phase of curiosity.

      Steven: …phase of curiosity.

      Sonal: Gotcha.

      Steven: And I love the word curiosity. But openness to experience is a slightly different way of thinking about it, that you are walking through life looking for, you know, “I’m open to this thing that I’ve stumbled across, and I want to learn more.” And Tetlock’s predictors — the superforecasters that we’ve talked about — they had that personality trait in spades in general. So it’s a wonderful thing, and it’s related, I think, to another quality which is empathy, right?

      Sonal: Mm-hmm, which is also, by the way, one of the very things that fiction helps with.

      Steven: Exactly, exactly. So when you get into the world of, kind of, personal decision-making, novels, in a sense, train the kind of empathy systems in the brain because you’re sitting there, like, projecting your own mind into the mind of another, listening to their inner monologue — their, kind of, consciousness — in a way that almost no other art form can do as well as a novel can. And so that exercise of just, “What would that other person think? What would their response be?” In so many decisions we have to make, you have to run those simulations in your head, right? Because your decisions have consequences to other people’s lives. And if you aren’t able to make those projections, you’re going to be missing some of the key variables.

      Sonal: That’s great. And then, finally, what do you make of all those folks that have, like, this list of tips and advice? Like, when they think about, like, “Jeff Bezos does this and Elon Musk does that.” I think you might have written about this in your book, about how Jeff Bezos believes that you should get to 70% certainty.

      Steven: Yeah, I actually — I like that technique, which is to say don’t wait for 100% certainty, because a lot of the challenge with these complex decisions is you cannot by definition be fully certain about them. So the question is, where do you stop the deliberation process?

      Sonal: So you don’t just freeze and not do anything.

      Steven: And by measuring your certainty levels over time, taking a step out of the process, and say, like, “Okay, how certain am I really about this?” I think that’s a really good exercise. So I think those little — you know, I definitely included them. I tried it with this book to try and hit the sweet spot of like, “These are kind of interesting tools that have been useful and that have some science behind them,” but also then to just look at the, kind of, broad history and some of the science about the way that people make decisions and somewhere have it kind of be a mix of those two things.

      Sonal: I think it’s great, and especially because we, as Homo sapiens, are very unique in being able to actually have the luxury of doing this. Well, thank you, Steven, for joining the “a16z Podcast.” He is the author of the new book just out, “Farsighted: How We Make the Decisions that Matter the Most.” Thank you.

      Chris: Thank you very much. It was great talking to you, Steven.

      Steven: I loved it. Thank you.

      • Steven Johnson

      • Chris Dixon is a general partner at a16z, where he leads the crypto/ web3 funds. Previously, Chris was cofounder & CEO of startups SiteAdvisor and Hunch (acquired by eBay); and an early blogger at cdixon.org.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      When Organic Growth Goes Enterprise

      Martin Casado, Andrew Chen, Russ Heddleston, and Hanne Winarsky

      What happens when the bottoms up, organic growth usually associated with consumer companies starts to go…. enterprise? Part of our continuing podcast series (you can listen to part one on user acquisition and part two on engagement/retention) on growth, this episode explores the increasing trend of enterprise growth shifting to be more “bottoms up” — with a16z general partners Martin Casado and Andrew Chen, and DocSend CEO and co-founder Russ Heddleston, in conversation with Hanne Tidnam.

      So what exactly does more bottoms up growth for enterprise look like? And then how does organic growth map into the direct sales model we traditionally see in enterprise? How does it affect company building overall? What changes in how we evaluate growth, what do we look at… and how can those two different models work best together?

      Show Notes

      • New growth patterns that are emerging [0:36], how DocuSign broke into the market [3:04], and the company profiles that most benefit from a hybrid approach to growth [4:39]
      • Important metrics that companies and investors should watch for [7:05]
      • When it’s time to hire a sales force [12:38] and other signs of rapid growth [14:55]
      • Pricing and packaging issues [17:43] and further discussion around when it’s time to bring in sales [21:20]
      • How new growth patterns are leading to organizational changes [25:15]

      Transcript

      Hanne: Hi, and welcome to the a16z Podcast. I’m Hanne, and today we’re talking about another aspect of growth. This episode is about the growth typically attached to bottoms up consumer companies, but that’s now more and more showing up in enterprise. So what does that more bottoms up growth for enterprise look like? How does it affect company building, how does it change how we evaluate growth, and what do we look at?

      Joining us to talk about the tactics and questions we should be thinking about in this kind of hybrid scenario are a16z General Partners Martin Casado and Andrew Chen, and Russ Heddleston, CEO and co-founder of DocSend.

      New patterns of growth

      Hanne: Let’s start with the super basic question, which is what exactly are you starting to see happen with this shift in enterprise?

      Martin: So traditionally in the enterprise, you’d build a product, and that product would be informed by your knowledge of the market. And then once that product was ready, you’d go ahead and sell it by hiring salespeople and the salespeople would go directly engage. You’d probably do some sales-led marketing where maybe the salespeople would go find the customers or you’d have some basic marketing to do it. But the majority of the go-to-market effort in the early days was this kind of direct sale.

      And we’re seeing kind of this huge shift, especially in SaaS and in open source where companies establish massive market presence and brand and growth using these kind of more traditional consumer-ish growth motions. And then that very seamlessly leads into sales, and often a very different type of sale. And so I think a lot of people in the industry are on their heels, both investors and people that have started companies in the enterprise before, they’re trying to understand exactly what’s going on.

      Hanne: Is it actually seamless? Is it a seamless transition there?

      Martin: Well, I mean, that’s often the question, right? So we’ve seen companies moving on either sides of this. Some companies are like, “You know, listen, we’re just going to do organic growth.” And they don’t actually do sales. And in our experience, these tend not to be kind of hyper growth on the revenue side. Right? So they’ll continue to kind of growth customers, but it’s hard for them to get these nice, hyper linear revenue growth.

      On the other hand, we see companies that will just do sales. And for them, it’s actually very difficult to grow quickly because they don’t have the type of funnel that you’d get from the growth metrics. And the ones that seemed to have figured it out the best, what they’ll do is they’ll create kind of a brand phenomenon. They’ll get this growth, they’ll get that engine working and then they do kind of tack on some sort of sales on the backend and then those two motions work in tandem.

      Russ: So if you’re a small startup, breaking into that big ACV sale is tough. You’ve got to have a really high annual contract value and everything is going to be more crowded. And it happens occasionally but it doesn’t happen as often. And if you’re trying to target a specific buyer, just getting access to them can be very challenging and that’s just a huge hurdle to overcome. Like, how on earth could anybody break into that? Consumer understands a lot of different tips and tricks because you have to be really frugal to acquire a customer that you’re just supporting with advertising to get someone who you make six bucks a year off of. You can’t spend any money to get that person. So there are a lot of tactics there that are really interesting. If you apply those to some of the B2B value propositions, you can actually break in in a way that no one else was really thinking about before.

      Hanne: Well, let’s get into those. What are some of those?

      Russ: The way we broke into the market is we took a relatively simple workflow which is sending content from one business to another business. And so we said, “Okay, a better way of doing that is to allow the person sending it to create 10 different links to the asset, send them off to 10 different companies and see what happens to them.” How long do they look at each page? Who do they forward it to? You can see what people care about.

      And so the first version of DocSend was just free. That actually just gets people using the product, and it’s cheap enough that they can keep everything else in their stack. So we’re not replacing anything, we’re purely additive at that point. And that’s really how we got our toe hold in the market.

      Andrew: Russ, how did you get your first 100 users?

      Russ: I think the first revenue we got was in the form of a bottle of whiskey that someone gave me as a thank you for giving them a account that they used for their own fundraising process.

      Hanne: What kind of whiskey?

      Russ: You know, I don’t actually remember it. I think the office consumed it relatively quickly so I don’t think it was around for very long.

      Andrew: But from a top of funnel standpoint, where did you get the first…

      Russ: It was all word of mouth. Forty-two percent of our signups are still word of mouth. Twenty-eight percent of our signups are from someone viewing a link and then getting interested and coming into the product.

      Andrew: when you look back at Dropbox the first thing they did to get traction was to announce on “Hacker News” and also “Dig” at the time was such a big deal, right? These days, maybe the actual platforms have changed, like, maybe you go to “Product Hunt” instead, maybe you go to Twitter. But ultimately, doing a big announcement but then kind of getting the all sort of viral word of mouth means that a lot of your first users end up experiencing it because one of their friends wants to show them the product, or they just decide they want to try it. As opposed to having somebody sort of email you or call you up.

      Hanne: Is there a certain kind of company that this works for better than others?

      Andrew: I think that there are certain kinds of products that can be all the way pegged to completely self-serve, bottoms up versus maybe what’s kind of in the middle. Is the product a horizontal enough product that literally you can bring almost all of your coworkers — things like Dropbox, Asana, Slack, these are all things that everyone in your company can use, and so naturally is going to spread much faster because at every moment, each node in the network is going to be able to have access to all 15 to 30 people around them where it can spread.

      The second thing is products that are actually really front and center in your workflows, all the acquisition that we see, especially virally, happens because of engagement. They’re deeply, deeply linked with each other. Because as you engage and as you’re using the product more, inevitably then you’re sharing links, you’re assigning tasks to people, you’re commenting on people’s files. These are all things that bring people back and bring new people into the product. There’s a whole class of products that aren’t completely horizontal that maybe only apply to a particular job title or function. And so that all of a sudden gets harder because maybe it can spread within the department, within the function, but it’s not going to go really broadly. And eventually you get to the set where it’s like, maybe there’s only a couple buyers in the entire company. And for that, you don’t go bottoms up at all. It’s just literally impossible.

      Hanne: So this middle zone is what we’re talking about, where there’s some indication but it’s not completely horizontal and viral. It needs a little bit layered on.

      Andrew: The new thing is that the fact that users can then bring these products into their workplace, and you might get a large company of 20,000 people with a patchwork of folks using a whole bunch of different products before IT actually makes a decision. Like, that’s new and very interesting.

      Russ: Every company tends to have some form of super power that’s available to it based on just what their business is and what their product does. So we typically add features in one of three buckets. One is to increase the spread of a business to another business. One is to get more lock-in within a company itself, so getting that spread within the company. And then the third is just making our customers more engaged. Because the more they’re using it, the more they’re sending it outside the company. Our top request at one point was, “I need to send a folder of content.” And you’re like, “Okay, that makes sense.” But what they really wanted was this kind of deal room thing. So we ended up building Spaces. And that just really increased engagement of our customers.

      Important metrics to consider

      Andrew: That is why one of the really interesting things that, Martin, you and I end up talking about with these bottoms up companies is evaluating the engagement on the products using consumer metrics. Because often, it’s the engagement that’s really the leading indicator for growth, but from an acquisition standpoint as well as retention, which then is sort of the leading indicator for, like, are they actually going to renew their subscription over time?

      Martin: So to me, this is one of the key questions. We see these companies that fall in between this kind of consumer-ish growth in this enterprise thing. And actually a question I’ve been meaning to ask you that I haven’t yet but this is a good opportunity, so is it the right thing to evaluate these things purely from a consumer lens? Are the growth patterns the same as you would see in consumer XX? Let’s even just put aside the question of sales. Should the growth metrics be the same as a consumer company?

      Andrew: When you’re evaluating even purely consumer products, you have to really look at what the expected behavior is. And so I would kind of turn the same question for the kinds of things we’ve been working on, which is obviously if you have users that are trying out some new email security product, let’s say, hopefully they’re not interacting with it that much. But if the whole pitch of the company is, “Hey, this is going to be the system of record for everything that your team’s going to work on for all of their projects, or whatever, and they’re going to use it every day,” then it’s like, “All right, then let’s actually start using, you know, daily active metrics in order to evaluate if that engagement is actually there.

      Hanne: What about from your point of view, Martin? Are there metrics that you…

      Martin: Well, yeah, I think it starts to get a little complicated. So there are a number of consumer metrics you track. One of them is engagement which gives you a sense of how often it’s used, and maybe that’s something that you can proxy to value. There also is just simply top of funnel growth, right? How many people know about it, what is the brand? The world I come from is nobody knows about the product when you start. There is no organic growth. Marketing is, at best, linear with the dollars you put in, the number of customers that are top of funnel, it’s probably sub-linear. All the value and monetization is driven my direct sales and so you’re…

      Russ: It’s account-based sales.

      Martin: It’s account-based sales. So your ACV has to be high enough to cover the marketing cap. So that’s one bookend. The other bookend is all of this growth stuff you do acquires tons of customers and then the product will monetize itself, right? So my big question is, is there a slider bar here? If you slide the slider bar all the way to the left, there’s the Atlassian model, and there’s very little sales. And if you slide your slider all the way to the right, then it’s just direct sales and no marketing. And then the question is, what does it look like in the middle? Because you look at it like the slider bar is all the way to the left, and I look at like the slider bar is all the way to the right. But more and more, we’re seeing companies that actually they’re very interesting on both sides, but they’re not classic on either.

      Andrew: Totally.

      Martin: So let’s assume we take the case of the slider bar as all the way to the organic growth and it’s purely horizontal and it’s growing like crazy. So the question is does it still make sense to build a direct sales force? As in, will it increase the unit economics if you do? I think our experience here with Slack and with Hub and with many companies is…

      Andrew: It’s definitely yes, right?

      Martin: Yeah, the answer is yes.

      Andrew: Because definitely yes.

      Martin: Because that’s how you maximize ACV per customer, because there is a procurement process and just finding the budget and maximizing that is something a human can do much better than a product at this point in sales.

      Andrew: Right, and in fact, I think actually even the virally spreading products end up going tilting towards enterprise over time for a really simple reason, which is that with larger companies your cohorts will look better because there’s revenue extension. Because no matter what, when you’re working SMBs, I find it very hard to get better than, let’s say, a 5% per month churn rate. All these little companies keep going out of business all the time, they’re fickle, they have small budgets, etc. And so what you quickly find is you have to go to the big guys, all the budget’s there. And so then that inevitably leads you, even when you’re completely bottoms up, to start building stickier products for enterprises and add the sales team, add customer service, and all of that. So I think that is the natural trend.

      Hanne: my question is when is that happening? Is that happening in tandem all along? Are they sort of naturally that hybrid from the beginning or do they slide along as things change in the company’s cycle?

      Martin: Specifically were you thinking about sales when you started?

      Russ: No. Not at all.

      Martin: The common refrain.

      Russ: When we launched DocSend, we didn’t have any background in B2B. So it kind of caught us by surprise and we got a lot of interest that we weren’t able to convert into dollars because we weren’t even charging people. If we could do DocSend over again, I think we could build it in half the time. Because I think this is a new type of company that there aren’t that many examples for.

      Hanne: if you were to put that very broadly as like the type of company you mean what is that type of company?

      Russ: If you create a business value, like a B2B value for something, you build some product and you release it for less money than you should or free, you’re going to get some usage of it. If you’re creating a B2B value, you kind of picked your target audience, you get your 100 accounts you want to sell it into, and you have people just pound on their doors to get in there.

      Martin: You literally start at the top of the list, you go to the bottom, and then you go back to the top of list.

      Andrew: And I think when you compare it to consumer…I mean, for most consumer audience-based plays, you really defer monetization for a really long time. Because you have to aggregate this huge audience and then you start talking about, like, okay, let’s look at ad-based models. And so, and you contrast that to these B2B products where you can actually monetize from early on. And in fact, when you monetize it actually unlocks a bunch of paid acquisition channels, and it’ll unlock sales, and it unlocks a bunch of stuff. I think that’s very confusing for people who, you know, they get started and they’re kind of in this consumer products mindset. And so they often end up kind of like, “Oh, how I do grow? How do I increase acquisition?”

      Inflection points to watch for

      Hanne: What are the signs that that’s the right time when it begins shifting, the sort of tipping point where you’re like, “Okay, should I need to pay attention to this?”

      Russ: We were just selling some small deals on the side. So I was like, “I think we should hire a salesperson.” So we hired our first SMB AE, and in our first month we’re like, “We don’t think she’s going to sell anything.” And she sold twice what the quota was supposed to be. There was just a lot of money laying around where if you actually talked to someone on the phone and explained it to them, they might have bought one seat before but now they’re going to buy 15.

      Martin: Didn’t you have a support collecting checks?

      Russ: We had a support person selling a lot of DocSend for quite a while.

      Martin: That’s a pretty good indication it’s time to do sales.

      Russ: Yeah, that’s another really indicator. Also, now that we’re going a little bit more up market, you actually need someone who’s able to run a good sales process even though they’re not doing the outbound part of it once you get them in the door, running a good sales process, having good sales hygiene, really understanding who your buyer is, you need to do all those things too. So you really need to marry both sides of it.

      Martin: Another shift I’ve seen, which is important from a company building perspective, so if you think about direct enterprise sales, the actual lead up to the sale can take nine months to 18 months. You’re working the account, you’ve got an SE in the account and you’re educating them, etc. So with these new companies, often the customer is education themselves, they’re already trying, and so much of the actual total value of the account comes after they’re users of the product. So it’s about expanding the account. So now there’s this very interesting relationship between sales and customer success where a lot of the value is actually being driven by customer success. I don’t think the direct enterprise is used to this model.

      Russ: Yeah, we always say, “You win the renewal when you do the onboarding.” And getting everyone engaged quickly with an account really helps with expansion and renewal. When we do onboarding, we have a little raffle. So if you’ve got 50 salespeople at your company and if you send a certain amount of DocSend links externally in the first two weeks, then you’re eligible in this raffle and you get one of three different prizes. It’s like a $200 bottle of whiskey or tequila or Amazon gift card. And that’ll actually…

      Martin: What kind of whiskey?

      Russ: I also don’t know. But that’ll actually get everyone using the product really quickly, and then they look at that and they say, “Oh, we bought the product for our sales team. Man, we should use this for our customer success team or our support team.” And so they build faith in it and then it naturally expands. Sometimes you need a salesperson involved, but more often than not, customer success is just saying, “Yeah, you can use it for that too.” And then they expand.

      Hanne: So I want to get into the timing question of when, when this starts happening. When you happen into this moment, when all of a sudden you realize, this would be helpful, how do you begin to actually make that happen? What are the signs and signals that are telling you now is the time?

      Andrew: Well, I think one really important one is what kinds of companies and people are signing into your service? Where you’re starting to see both prominent tech companies as well as Fortune 1000s just signing up to try it. Even on a purely bottoms up basis, you create the funnel from signing to using a contact enrichment service and starting to score all of these new users that are coming in. And if you find out that a large proportion of them are actually enterprises, that’s actually pulled demand from the market that you should actually be up leveling faster.

      Russ: One of the things we actually did to spread that awareness faster is we decided that marketers will send off tons of things to people, so why don’t we just support the marketing use case? Not because we make more money from that. If we power, for instance, a researcher port for a company, they’re sending that to tens of thousands of people that then get exposure in lots of areas that we weren’t even in before. So it really kind of allows it to hop into other places, and then we generate more of that demand coming in. You need to take a look at who’s signing up for your product and you need to think about what might they be looking for and what problems might we be able to solve for them?

      Andrew: Another thing I might add is what kinds of feature requests folks are having. If you’re building something that’s like an email client, something that is really horizontal or it’s a new document editor, everything’s great and all of a sudden, you start getting these future requests for Salesforce integration, and you’re like, oh, okay, this is like a different…

      Russ: Another request we’ve always gotten has been DocSend, you can’t actually send anything from DocSend and it’s really nice to be able to send from email and customize it, and there’s a different philosophy around that but we were thinking, like, “Man, just let people send stuff right from DocSend. Because then it’s got a DocSend email that they get.” And so it’s actually a good growth thing, as well. So you can, kind of, reprioritize your product list based on how much it’s going to spread awareness about your product outside of the company, which is a great lens for every company to use when thinking about trying to make these viral loops go faster.

      Hanne: That’s interesting. Okay, so say ideally you do have this kind of blended model going on. Are there conflicts ever in the types of information that you’re getting from the different sources?

      Martin: At the highest level, I think there actually are a lot of conflicts in these motions and in a number of areas. And the most obvious one and this is something that’s so prevalent in open source is, a good way to get organic growth is to give something away for free. And if you give it away for free, it may be hard to monetize it because a lot of the assumptions here are predicated on organic growth, there’s always an open question of how much do you give away versus how do you monetize it? Enterprise really is all about monetization because there is no conversion between eyeballs and dollars like you do in kind of more advertising-like domains. And so there’s a real tension there.

      Pricing and packaging

      Hanne: So how do you think about that balance?

      Andrew: It’s sort of funny because it sort of implies that you can go one way and not the other. Meaning, if you have a product that’s making a bunch of money and you have a highly functional sales team, and then a product person in the org is like, “Hey, let’s have a free offering,” that is not going to happen. Versus the other way where you have something that’s product led and it generates a lot of users and then you build this whole pipeline off of that and you build the sales org. If you do it in that order, all of a sudden the freemium product actually feels like it’s actually very helpful. Nevertheless, eventually free tends to go away or become pretty crippled as the whole business evolves. But freemium can be so disruptive in these industries because if you’re a large enterprise, B2B software company, you’re not going to be able to do this kind of low end free offering.

      Russ: Yeah, a lot of what we’re talking about is just pricing and packaging which is something that’s so hard for everybody. because you’ll look at a company and you’ll look at their pricing and packaging, and you’ll be like, “Congratulations. You’ve done it.” But then when you look at a new company and be like, “What should their pricing be?” Everyone’s like, “I have no idea.” And it’s hard because you can’t AB test it. And so you have examples of what’s worked but it’s really hard to predict what will work for any given business and so you could say on the low end, we got a free thing. On the high end, we got an enterprise thing. And then maybe there’s something in the middle.

      We actually just increased the pricing and added a couple new plans. And we thought the conversion would come down but we’d make more money. What happened was that conversion went up and we made way more money.

      Hanne: And why do you think that was happening?

      Russ: We moved some features around and then we talked about the plans differently and who they’re for. And so people also trusted it a bit more because they’re paying more for it. People then value it more and actually use it more because they’re paying for it.

      Andrew: Right. Well, I mean this is the difference between also when Netflix increases their monthly subscription by $2, everyone’s screaming bloody murder. And B2B is obviously less elastic.

      Hanne: “Oh, it must be good.”

      Andrew: There’s some price signaling as well.

      Martin: But it’s also important to compare it to traditional pricing and packaging. the general model used to be when you first come to market, you are as expensive as possible and you know you’re going to go for a limited set, but ACV is high enough to cover it. And the sales cycles are long anyways. And then after you feel like you’re saturating that, you offer lower priced units so that the aggregate market is larger net cannibalization. So you don’t want to cannibalize yourself. And the way you do this is market research of existing customers, you know the target customer base, and you can AB test. You can actually do fairly small rollouts because it’s not marketing led.

      That motion is lost in this world because basically, as soon as it’s publicly available for free, everybody knows about it and it’s very difficult then to kind of retract that. So you have to be very thoughtful about pricing and packaging upfront because any experiment basically is reality now. And that’s very, very different from the traditional enterprise motion. I mean we experimented with pricing so much in the early days and the only thing you had to hold sacrosanct was price very expensive early on because you’re only going to get 10 customers anyway and you just can’t do that motion now.

      Andrew: Even the way that you do pricing, it can potentially impact engagement. Where do you put your pay wall? Is it a time-based trial, is it a usage-based thing? Those things become really important because, especially when you have a product that is growing virally, it’s building a network inside these companies, you don’t want to cut off the network prematurely, because the network is what makes the whole thing sticky. So for example, it would not make sense for a product like Slack — if they were like, “Well, we’re going to cap the number of people that can join the channel to five,” that doesn’t make sense because the entire network effect is based on having all of your colleagues there. So what you end up wanting to do is you’re gaining these features that the IT admins want, and those are the things that end up being how you differentiate the enterprise customers from purely the consumer ones.

      When to focus on sales

      Hanne: When you start thinking about forecasting or planning, do you ever get competing signals and information from this blended model where you’re doing two different kinds of growth and sales?

      Martin: Well I think this is a really interesting question of…for wherever you are in the lifecycle of the company, let’s say you have $1 to spend on go to market, how much of that $1 goes to brand and marketing, versus how much of that $1 goes to sales? And that is a question I don’t think anybody knows the answer to.

      Hanne: But what are some of the ways you start figuring it out?

      Martin: The traditional view in the enterprise is you spend it all on sales, basically, until you’ve got a working pipeline or a repeatable sale. Then you have economics you understand and then you start increasing the top of funnel. That’s the traditional model. But now, we’re marketing led. And so, how do you know how to split those dollars up and when to do it?

      Russ: A lot of it has to do too with the DNA of the founding team. my two co-founders and I are all engineers and product people. And so we’ve basically used our product as the marketing engine for the company so far. We haven’t done any paid acquisition, we haven’t been doing a lot of marketing stuff that’s been driving a lot of the top of the funnel. The product itself is driving the top of the funnel.

      Hanne: But that would be what most of these companies are doing kind of? In this kind of company, that would be common?

      Martin: Well, okay, I mean there are a number of companies that will actually just buy their users. I’m totally not used to that. Andrew’s totally used to that. And so this is kind of…

      Andrew: …Yeah, and I hate it. Yeah, there’s folks that they’re spending tons and tons of money on Facebook, on Google, etc. That’s very common. The other one as well is a huge focus on content marketing as being one of the primary channels I think that is really different.

      Russ: It’s kind of going back to what we said earlier where, should companies invest in sales? And my view on that would be, if you show me a company that’s growing organically, I’ll show you a company that’s performing better if you also add a sales team to it. If you can get it working with the product, you can actually probably get a good baseline of growth, but you should probably spend more on marketing and sales on top of that. And if you can get the unit economics anywhere near reasonable for a paid acquisition, you should probably put everything you can into that channel, knowing it’s just a component of your overall strategy.

      Andrew: The thing that makes it hard to normalize a bunch of these efforts is they happen on very different time scales. You can literally increase your paid acquisition budget and see a spike in signups and self-serve conversions within a 24-hour period. If you’re going to go and hire and build out your sales team, it’s going to take you months to build the team, and then months to recruit them. But when the revenue hits from these really large contracts, it’s huge. Hopefully, you have multiple systems that are mutually reinforcing each other as opposed to feeling like they’re in conflict. But that certainly happens if you are trying to figure out, where do I put the next dollar?

      Hanne: I mean, what are some ways around dealing with that discrepancy between timeframes and planning and forecasting when you’re trying to match up these two very different chronologies?

      Martin: I don’t think there’s any recipe. There’s never a recipe to doing a startup anyway. There’s no recipe to find product market fit. I don’t think there’s any recipe to knowing what’s the right balance between growth and sales and when to do it. But here are things that a founder should think about that has traditional enterprise expertise in the new world. The first one is brand. You normally don’t think about brand, but brand does drive viral growth. Product focus, right? The product itself actually creates virality. The enterprise very rarely thinks about, believe it or not, product. They think more about solving problems.

      Hanne: Really? That’s so surprising.

      Martin: It’s not about making the product “delightful” or easily consumable. It’s solving a real problem and adding business value and less about consumability, right? Now you have to think a lot more about consumability, like single-player mode, like self-service mode. Right? Very different than traditional enterprise. You need to design your company for bottoms up growth whether you’re open source or you’re doing SaaS or whatever, because this is the new method of consumption. And I do think that the one most important is if you’re doing bottoms up growth, I think you have to expect a lower ACV which is a different way to build a sales team. And so you just have to be more comfortable with your inside, inside/outside models and then you have to be more comfortable with focusing in on expansion rather than upfront ACV.

      So these are all very, very different than the traditional enterprise.

      Hanne: They’re sort of mind shifts.

      Martin: They’re all mind shifts.

      New organizational structures

      Andrew: There are new organizational structures that end up being built within these companies that sit alongside sales because all of a sudden, you can have multiple revenue centers, right? And that’s a very different approach. Then the people that you hire for this end up being designers and PMs and engineers that are kind of this business-y, metrics focused folks. Going back to Dropbox, I know the most recent incarnation were sort of biz ops people turned PMs that were previously working oftentimes in consulting or banking.

      Hanne: So it’s a new hybrid kind of role in organization as well that comes down from this?

      Andrew: Right, exactly.

      Hanne: That’s interesting.

      Andrew: Do you want to hire the nth engineer into this team that can run a whole bunch more of these AB tests? Or do you build out your sales team more? These are the kinds of decisions that these companies have to make these days.

      Hanne: Russ, did you see that as well that kind of hybrid role?

      Russ: Yeah, there are a lot of things that aren’t just salespeople calling and getting contracts signed. Enterprise sales is like a playbook that makes sense. For the bottoms up company, you’ll see this perfect curve and kind of the outside view of that is they did something brilliant at the beginning and then everyone went on vacation and it just kept growing. But in reality, behind the scenes is a series of every smart things you did to keep that growth going. And what got you from A to B is not going to get you from B to C. So you often have to to redo your organization, you have to add in new roles, and you have to recognize when you’re going to hit points of diminishing return for a type of investment. And you have to get ahead of that and say, “Well, what’s the next type of investment we’re going to be able to do to get us to the next stage of things?”

      Hanne: Add on another layer, right?

      Russ: Right.

      Hanne: As Jeff would say.

      Russ: Yeah, it’s different for every company. There’s no one right answer.

      Andrew: The really important key thing is the importance of not just a great product but literally great user experience and design, and all the fit and finish that you would expect with a completely modern consumer-facing application.

      Hanne: Now that’s coming to this world too.

      Andrew: Right, exactly. Like, Envoy, that is an amazing B2B viral story. They’re very rare, But the reason why people use that now is because offices are part of the brand experience. And then after they use the thing, then they’re kind of like, “Oh, yeah, we’re using pen and paper back at the home office. We need to upgrade to this too.” These examples crossover both the consumer sort of design world, all the way to sales, all the way to performance marketing. You really have to leverage a lot of skills in order to execute these strategies.

      Russ: The expectation for the usability of software I think is going up in enterprises. Larger companies expect more polish and more usability. And if it’s not there, they start to really worry about it being shelf-ware or not the value proposition. And shelf-ware is a pretty big problem at a lot of big companies.

      Andrew: One of the funny anecdotes at Uber was that for a long time, we were officially on Hip Chat but there were so many teams across the company that would have their little secret Slack team chat going because they just didn’t wanna…

      Hanne: Illicit Slacking?

      Andrew: I feel comfortable saying that now that Hip Chat’s been shut down. Employees will literally rebel and use whatever they want. And so as a result, as companies selling into these, your products have to be really good to compete with everything else that’s out there.

      Martin: I didn’t understand how powerful actually just growth tactics were. independent of product. Actually independent of sales. Andrew, you and I were looking at a company which was amazing. Like the growth was amazing. Like all of these numbers were amazing. The engagement, they were monetizing, like everything looked great and the conclusion we came to was, like, it’s because they just had, like, such an amazing growth team that was almost independent of the product that they were selling.

      Hanne: Oh my gosh.

      Martin: We literally came to the end and we’re like, “Wow, this could be anything. This could be, like, you know, dog food. This could be, like…

      Hanne: Doughnuts.

      Martin: Yeah, whatever if you figure out how to do it right, it’s a very, very powerful thing. And by the way, that used to be what you said about sales. What you used to say about sales is if you have a very good sales team that understands the buyer, you know, it’s kind of independent of product.

      Hanne: Awesome. Thank you guys so much for joining us on the a16z podcast.

      Group: Thank you.

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      • Andrew Chen is a GP at a16z in consumer technology. Prior to that, he led Rider Growth at Uber. He has written for a decade on metrics & growth on his blog/newsletter, and is the author of The Cold Start Problem.

      • Russ Heddleston

      • Hanne Winarsky

      The Basics of Growth, Engagement, and Retention

      Andrew Chen, Jeff Jordan, and Sonal Chokshi

      Once you have users, how do you keep them engaged, retain them, and even “resurrect” or re-engage them? That’s the focus of this episode of the a16z Podcast, which continues our series on the basics of growth from user acquisition to engagement and retention — covering, as always, key metrics and how to think about them. Especially as many products and platforms evolve over time, so do the users, some of whom may even use the product in different ways… so what does that mean for engagement, and how can startups analyze their users? “Show me the cohorts!” may be the new “show me the money”…

      Featuring a16z general partners Andrew Chen and Jeff Jordan, in conversation with Sonal Chokshi, the discussion also covers everything from how network effects come in to play (is there really a magic number or “aha” moment for a product?) to who are the power users (and the power user curve for measuring, finding, and retaining them). Because at the end of the day, you don’t want a leaky bucket that you’re constantly trying to fill up. That doesn’t work, and definitely won’t scale.

      Show Notes

      • Discussion of engagement and natural trends, including how to measure both [1:03]
      • The importance of analyzing cohorts [5:39] and identifying the “aha” moment [8:44]
      • Moving users up the ladder of engagement over time [10:55], further discussion of measuring engagement (DAU/MAU) [15:39], and the reliability of certain measurements [19:56]
      • Network effects [22:04], engagement vs. retention [24:50], and how to measure retention [29:11]
      • Advice for companies [31:07]

      Transcript

      Sonal: Hi, everyone. Welcome to the a16z Podcast. I’m Sonal. Today’s episode continues our series on growth — the first part covered the basics of user acquisition — and so this part covers, more specifically, engagement and retention. Including, as always, key metrics and how to think about them.

      Joining us to have this conversation, we again have general partners Andrew Chen and Jeff Jordan. And we cover everything from how do network effects come in to is there really a magic number or aha moment for a product? To who are the power users and what is the power user curve for measuring them. But first, we begin with what happens after the initial acquisition phase, as different kinds of users join a product or platform over time — what does that mean for engagement; and how do you analyze them, using cohort analyses?

      Strategies for measuring engagement

      Andrew: One of the things that you see is that people end up using these products very differently. Because the kinds of users that you’re getting are changing over time. When you look at something like rideshare, you know, all the early cohorts are basically people in urban areas. And in these days all of rideshare is more like suburban or rural folks because you’ve saturated all of the center. And so what you tend to see is as you acquire your folks, your core demographic out that actually ends up showing up in the engagement.

      And so, you know, going back to a natural “gravity” to the whole thing [discussed in episode one], this gravity also hits the engagement side of things as well — and then ultimately the LTV because your users were typically getting less valuable. I may take years to see this kind of play out but that’s kind of the natural law of things.

      Jeff: There is a progression in these and particularly the ones that are really successful. Early on it’s all about getting users. <Sonal: Right.>  And it’s just like users, users, users. If you’re widely successful at doing that you run out of users (or you start running low on users) and you have to go to engagement. So Pinterest has a very high-quality problem right now. Most women in America, have downloaded the Pinterest app.

      Sonal: Oh yah, I’ve had it for years.

      Jeff: Some growth can come through, okay, there’s some number of women who never heard of Pinterest somewhere in the country. But much more so they need to engage and re-engage the existing audience. I mean, we love engagement from an investor standpoint because it’s just, you know, that [crosstalk]

      Sonal: [crosstalk] It shows stickiness.

      Jeff: You can often hack your way into new users. It’s really hard to hack your way into true engagement. <Sonal: Keeping them.> Someone is spending 20 minutes a day on your site. Offerup, Pinterest the major investment thesis was, “Oh, my God!” look at that engagement … And, you know, if they can scale the userbase it’s a beautiful thing.

      Sonal: Right. What we mean by engagement is actually interacting with them and seeing their activity. Because to Andrew’s three points of acquisition, engagement, retention, the third piece is keeping them.

      Andrew: The way that we’ll often analyze this is looking at cohort analysis.

      Sonal: Yes.

      Andrew: Where we’ll look at kind of each batch of users that’s joining in each week and really start to dissect like well, how active are they really and to compare all these cohorts over time. You’re basically putting the users that come in from a particular timeframe, let’s say it’s a week, and you’re putting them into a bucket, right? And what you’re doing is you want to compare all of these different buckets against each other.

      And so what you typically do is you look at a bucket of a cohort of users and you say, “Okay, well, you know, once they’ve signed up the week after, how active are they?” And what about the week after that and the week after that and you kind of like can build out a curve. And it just turns out that these curves once you’ve looked at enough of them surprisingly, human nature, they all look kind of the same. They kind of all kind of curve down and for the good ones they start to flatten out and plateau and then, for the really good ones they’ll actually swing back up and people will come back to the surface. What you want to do is you want to compare the various cohorts against each other in time to see if you can spot any trends on how the usage patterns are, increasing or decreasing. When you add a new layer to a layer cake, you might unlock a bunch of new behavior. You might unlock a bunch of new frequency that didn’t exist before. Or alternatively, over long thresholds of time, people tend to become less active as you move out of your cohort segment.

      Sonal: The cohort graduates.

      Andrew: Whether or not a specific cohort of users flattens out is really important, right? Because, you know, if you’re in a world where they kind of slowly degrads and then all of a sudden it’ll actually go to zero, that means that you’re always kind of filling up the bucket — You have a leaky bucket, you’re constantly filling it up.

      Sonal: You’re always filling it up. Right.

      Andrew: Right, and what happens is that gets progressively harder because, if you want to keep your overall growth rate, because that means you have to double, triple, quadruple your acquisition in order to counteract for that.

      One growth accounting equation that’s often thrown around is that you know your incremental — your net — MAUs, right? So your net monthly active users equals all the new people that you’re acquiring, minus all the people that are churned, and then plus all the people that you’re resurrecting…

      Sonal: …Re-engaging.

      Andrew: Re-engaging, exactly, that are coming back after they’ve churned. And so what happens is for a new startup you are completely focused on new users because you don’t really have that many users to churn, and over time as you get bigger and bigger and bigger what you find is that your churn rate starts to — it’s a percentage of your actives.

      And so the evolution of most of these companies as they’re getting bigger tends to start with acquisition, then focus much more on churn and retention, and then ultimately also to layer in resurrection as well.

      The value of cohorts

      Jeff: And the cohort curves have a couple of other features that I love. Usually in marketplace businesses, the best models are built off of the cohort curves.

      Sonal: Interesting!

      Jeff: Because you have to understand that degradation and where it goes. Using cohorts really give you a sense of their network effects, and network effect is the business gets more valuable to more users that use it; if it gets more valuable, your newer cohorts should behave better than your early cohorts.

      Sonal: Why is that?

      Jeff: Because the service is more valuable given how they are.

      Sonal: Interesting. So that’s kind of a tip–

      Jeff: So in OpenTable if there’s ten times more restaurants you’re going to get a whole lot more reservations per diner because you were serving more of their needs. The OpenTable cores would climb up and get more attractive over time versus, you know, we talk about typically they tend to degrade over time. If you’ve reversed the polarity and they’re growing over time it means you’ve made the business more valuable. And then you start projecting forward. Okay [crosstalk]

      Sonal: What a better way to know the business is actually more valuable than thinking it’s valuable and believing your own myth.

      Jeff: In a network effects businesses we always ask, show us the cohorts. Everyone is [inaudible] on network effect, I’m a network effect But, you know, when you say, “Show me the data, cohort curves, or [crosstalk].” They don’t like it.

      Sonal: It’s like show me the money, it’s now show us the cohorts, I get it.

      Jeff: They don’t lie.

      Andrew: The other really interesting part is segmenting it.

      Sonal: I was about to actually ask you what are “the buckets” of cohorts? Are they all demographic data?

      Andrew: For a bunch of hyper-local type businesses, the reason why segmenting it based on market geography, why that’s so valuable is because then you can compare markets against each other. You can say, “Well, you know, this market which is like, has much more density in terms of the numbers of scooters behaves like this.” And you can start to draw conclusions, sort of a natural A/B test in order to do that.

      And I think the similar kind of analysis you can do for B2B companies is for products that have different sized teams using it. If you have a really large team that they are all using a product, well, are they all using the product more as a result? And let’s compare that to something that maybe only has a couple. … And so this way you can start to kind of disassemble the structure of these networks and do they actually lead to higher engagement.

      Jeff: Slack would be a perfect example of that, you know, just if you have five people in the organization using Slack you get one use curve. If you have the organization it’s the operating system for the organization; you have a very different curve.

      Sonal: Though it’s not just an accident, you have to sort of architect it, not just expect, like, serendipity to fall into place.

      Andrew: So after you get the new users, the way that you have to think about it is around quality, right? You have to make sure that the new users turn into engaged users. One of the things people often talk about is just sort of this idea of like an “a-ha” moment or a magic moment where the user really understands the true value of the product. But often that involves a bunch of setup. So, for example, you know, for all the different social products (whether that’s Twitter or Facebook or Pinterest, etc.), you have to make sure that when you first bring a new user in, they have to follow all the right people. They have to get, you know…

      Sonal: It’s like the onboarding experience.

      Andrew: …which, by the way, isn’t just signing up but it’s actually doing all the things to get to this a-ha where you’re like, “Oh.”

      Sonal: “I get this product.”

      Andrew: I get this product. It’s for me, And once you get that, then they’re kind of, you know, then you have the opportunity to keep them in this engaged state over time.

      Sonal: Is that really such a thing that there is, like, an a-ha moment? Or is it sort of like a cumulative… a lot of the later users on Facebook came because everyone else was already there. Is this only tied to new users?

      Andrew: In the case of Facebook actually, the fact that everyone was already there makes the a-ha moment that much more powerful, right? Because all your friends and family, they’re already there; your feed’s already full of content. And the first time that you see photos that maybe, you know, someone that you went to high school with, right? That is like whoa.

      Sonal: That’s actually what happened to me. I was so excited when I saw an old friend, right?

      Andrew: Right. Yeah, exactly. And so what that means is, you get the product and then afterwards, when you actually are getting these push notifications or emails that are like, “Hey, it’s someone’s birthday,” or whatever, you’ve internalized what that product is. And, you know, this moment is different for all sorts of different companies.

      Jeff: I’ve always heard this referred to as the magic number. When you show up and it’s a blank slate, it’s like, “What is this about?” But they would drive you maniacally to follow people because when you got to their magic number where they had statistically correlated the number of followers with long-term engagement and retention — they would kill you to get you there, doing what felt like unnatural acts of, like, you log on, follow, and you say no, and they say yes — but when they got you there, it kicked in, and the service then quote/unquote worked for you.

      A lot of the entrepreneurs I work with are trying to figure out what is my magic moment that then creates the awareness of the value prop. So take the example of Pinterest. Pinterest when it goes into a new market, first of all, they figured out they need a lot of local content to make it compelling to local users. The U.S. corpus of images doesn’t necessarily…is helpful in international markets but isn’t sufficient. And so they needed to supplement…

      Sonal: …You’re right. If I’m Indian, I want, like, saris. I don’t only want, like, skirts, which I may not be able to wear in certain regions.

      Jeff: Yeah. Exactly. I haven’t worn a sari in North America in a long time. <team laughs> But then once you have the content set, then you have to get compelling information to that individual in front of them, which, you don’t know the individual when they walk in the door, the faster they do that, the more quickly, the better the business performs; engagement goes up; retention goes up; and it works. So different entrepreneurs had to figure out what is that…what experience do they want to deliver where people get it? And then how do you engineer your way into delivering it?

      Increasing engagement

      Sonal: Okay. So we’ve come up through acquisition and you’ve gotten new users. They get the product. You even hopefully have a way to measure that and see and track it over time. Do you want then go into trying to get different users? Do you take your existing users? One of the things that we covered very early on is that with SaaS, you always wanna try to take existing users and upsell them because it’s way more expensive to acquire a new customer in that context. (I mean, of course, you wanna grow your customers.) How does this play out in this context? What happens next?

      Jeff: In a lot of companies, it’s a progression. So almost all the early activity in a company is, “Okay, how do I get the users?” As you get users, you get more and more leverage from efforts at activation and retention and engagement. So, I mean, use Pinterest as an example: again, a very high percentage of women in America have downloaded Pinterest. Then the leverage quickly goes into, “Okay, how do I keep them engaged? Reactivate the ones who disappear?” And their acquisition efforts in the U.S. get de-emphasized and all the leverage is there except as they’re going international, they’re still in that acquisition part of the curve. And so I think the leverage changes over time based on the situation of the company. Facebook hasn’t had any users in the U.S. in forever because they have them all.

      Sonal: This kind of goes back to this portfolio approach to thinking about your users.

      Andrew: Once you have an active base of users and customers, what starts to get really interesting is to really analyze what are the things that actually set that group up to be successful really long-term sticky users versus what are the behaviors and profiles where users aren’t successful, right? You actually throw your data science team on it to figure out all the different weights for both behavioral as well as the demographic and sort of profile-based stuff on there. And so one of the first things that you figure out is that each one of these products actually has this ladder of engagement where oftentimes new users show up to do something that’s, valuable but potentially infrequent. And you need to actually level them up to something that happens all the time.

      For example, when you first install Dropbox, the easiest thing that you can do is you can use it to just sync your home and your work computers, right? And that’s great but really the way to get those users to become really valuable is for them to share folders at work with their colleagues. Because once they have that and people are dragging files in, and they’re really starting to collaborate on things, that’s like the next level of value that you can actually have on a daily basis versus this thing that kind of is in the background that’s just syncing your storage.

      Sonal: So what are some of the things that people can then do to move those users up that “ladder of engagement”?

      Andrew: Step one is really segmenting your users into this kind of engagement map, oftentimes you’ll see this visualized as a kind of state machine where you have folks that are new, you have folks that are casual, and you want to track how much they’re moving up or down in each one of these steps.

      And then once you have that, then the question is, okay, well, great, how do you actually get them to move from one place to the other? First there’s like content and education; they need to know in context that they can actually do something. So for example, if you can get your users to set their home and work for a transportation product then you can maybe figure out, okay, should I prompt them in the morning to try a ride based on what the ETAs are?

      Sonal: Like in the app, there would be some kind of notification.

      Andrew: Like lifecycle messaging kind of factors in there. The second is of course if your product has some kind of monetary component, then you can use incentives like $10 bucks off your next subscription if you do this behavior that we know for sure gets you to the next step. And then the third thing is really just like refining the product for that particular use case, maybe there are certain kinds of products that are transacted all the time and so you maybe want to waive fees or you give some credits or you do something in order to get people to get addicted to that as a thing.

      Jeff: The really interesting thing is the frequency with which something is consumed. I mean, eBay had enormous levels of engagement early on for an ecommerce app in particular. People would spend hours just browsing because early on it was about collectibles and it was about people’s passion. So if someone’s passionate about Depression-era glass, they will spend hours if you give them that depth of content to say, “Oh, my God. I just found the perfect item.”

      OpenTable and Airbnb are both typically much more episodic. Most people don’t dine at fine dining restaurants with high frequency; our median user dined twice a year on OpenTable. And so that has completely different marketing implications and user implications. Measurement is probably even more important in the engagement/retention thing because we got our data scientist to understand the different consumer journeys through our product, and then we tried to develop tactics to accelerate the journeys we wanted and limit the journeys we didn’t want. But in order to develop your strategy, you really need to understand how your users are behaving at a really refined level.

      DAU/MAU and other metrics

      Sonal: So what are some of the engagement metrics?

      Andrew: One really important area is frequency, like, just how often are you using the product regardless of the intensity and the length of the sessions and all that other stuff. Literally just frequency of sessions. We might often ask for a daily active user divided by monthly active user ratio, and that gives you a sense for how many days is a user active?

      Jeff: DAU to MAU.

      Sonal: You recently put a post out on the DAU/MAU metric.

      Andrew: Right.

      Sonal: And when it works and when it doesn’t. There’s a lot of nuances around when to apply it and when not to.

      Andrew: DAU/MAU was very much popularized by the fact that Facebook used it, including in their public financial statements, and it really makes sense for them because they’re an advertising business and it matters a lot that people use them a lot all the time.

      Sonal: It’s like counting impressions and being able to sell that to advertisers.

      Andrew: Exactly, their products have historically been 60% plus daily actives over monthly actives. And that’s very high. You’re using it more than half the days in a month. On the flip side, what I was talking about in my essay about this is that DAU/MAU can tell you if something’s really high frequency and if it’s working, but a lot of times products are actually lower DAU/MAU for a very good reason because there’s sort of just a natural cadence, you know, to the product. Like, you’re not gonna get somebody who is using a travel product to use it more than a couple times per year. And yet there are many valuable travel companies, obviously.

      Sonal: So you’re saying don’t live and die by that alone.

      Andrew: Exactly.

      Sonal: Because it really depends on product you have, the type of nature of use it has, etc.

      Andrew: You just want to make sure that the metric reflects whatever strategy that you’re putting in place. If you think that your product is a daily use product and you’re gonna monetize using a little bit of money that you’re making over a long period of time but your DAU/MAU is low, is like sub 15%, then it’s probably not gonna work.

      And then a metric called L28, which is something else that was pioneered certainly early at Facebook: It’s a histogram and what you want to do is —

      Sonal: — A histogram is a frequency diagram.

      Andrew: Right. A frequency diagram that basically says, okay, show a bar showing how many users have visited once in that month, then twice in the month, and then three times in the month, and then four times in the month. And you kind of build that all the way out to 28 days.

      Sonal: Because there’s 28 days in the month on average.

      Andrew: And the 28 days is to remove seasonality and then a related one obviously is like L7, right? So just like last seven days. And so what you want to see…

      Sonal: So would this be WAUs (“wows”)? Weekly active users? Is that really a thing, by the way? Or am I just making that up?

      Andrew: Right. WAUs, DAUs over WAUs.

      Jeff: You just coined it.

      Sonal: I know. Great. I coined retainment. Why not?

      Andrew: Right. And so the idea with L28 or an L7 is the idea that you should actually start to see when you graph this out that there’s a group of people who just use it 28 days out of 28 days, right? And that there’s a big group of people who use it 27 days out of 28 days, and that there’s a big cluster. And so that’s how you know that you actually have a hardcore segment. And that’s really important because in all these products you typically have a core part of the network that’s driving the rest of it, whether that’s power sellers or power buyers or, in a social network the creators vs. the consumers.

      Jeff: I actually have heard this referred to as a smile because the one use is always pretty big. A lot of people show up once, “I don’t understand what this is,” and disappear… And then they typically slide down, more people use it…fewer people use it two days than one, three days than two. Done right, it starts to increase at the end. So you basically get a smile. You just go down. And I mean, that’s really powerful. Facebook had a smile. WhatsApp had a smile. Instagram had a smile. If you take a step back, it’s a precondition for investing in a venture business essentially that there’s growth. If it’s end market you want to see growth, but growth by itself is not sufficient. Investors love engagement. So Pinterest, the key driver of Pinterest, it was growing but the users were using it maniacally.

      Sonal: Oh, my God. I think I spent an entire Thanksgiving using Pinterest.

      Jeff: It was the engagement that blew my mind much more than the growth. OfferUp has engagement that’s similar to social sites like Instagram and Snap. I mean, a ecommerce site, you know, mobile classifieds, people just sit there and troll looking for bargains, looking for interesting things.

      Sonal: It’s a little addictive to see what’s nearby that you can buy. Why not? Yeah.

      Jeff: So DAU to MAU, smile, all these metrics are so core to us because a big engaged audience is so rare and, as a result, it’s almost always incredibly valuable.

      Andrew: And the engagement ends up being very related to acquisition because when you look at all the different acquisition loops — whether it’s paid marketing or a viral loop or whatever — all of those things are actually powered by engagement ultimately. Like, you need people to get excited about a product in order to share content off of that platform to other platforms in order to get a viral loop going. And so one of the things I was gonna also add on DAU/MAU and L28 is that they’re actually really hard to game, right? Which is fascinating.

      Sonal: Yeah, why is that?

      Jeff: Whereas growth can be very easy to game.

      Andrew: Right, exactly.

      Sonal: Why is that? What’s the difference?

      Andrew: The typical approach is to say, “Well, you know, I’m gonna add in email notifications. I’m gonna do more push notifications. I’m gonna do more of this and that.” And then automatically, you know, these metrics ought to go up, right? The challenging thing is actually usually sending out more notifications will actually cause more of your casual users to show up because your hardcore users were already kind of showing up already. And what that does is that’ll increase your monthly actives number but actually not increase your daily actives as much. So I’ve actually seen cases where sending out more email decreases your DAU/MAU as opposed to increasing it.

      Sonal: That’s really interesting. When you think about this portfolio of metrics, it really tells you a story about people are kind of coming but not really staying–

      Andrew: If you get an email or a push notification every day, eventually you turn them off, and then you just stop. So then you get counted as a MAU for that period of time and then you lose them as a DAU. Acquisition is super easy to game because you can just spend money.

      Jeff: Or you’ve got a distribution hack that works. Early on in the Facebook platform, companies literally got to a million users and it felt like minutes. Just because there were so many people on Facebook and the ones who were early just got exploding user bases. There were a number of concepts whose mean number of visits was one. They never came back. So you get to see these incredibly seductive growth curves but our job is essentially to be cynical and just say, okay, we need to go be it below that because there are a lot of talented growth hackers who can drive growth. I looked at a number of businesses that had tens of millions of users and no one ever came back.

      Sonal: This is why engagement is so, so key.

      Network effects

      So we’ve talked especially about the fact that growth and network effects are not the exact same thing. Because network effects by definition are that a network becomes more valuable the more users that use it. What happens on the engagement side with network effects? What are the things we should be thinking about in that context?

      Jeff: Typically network effects, if they’re real, manifest in data. Things like the cohort curves improve over time. Usually there’s a decay. With network effects, there often is a reversal where they’re growing because it’s more valuable. Another smile, essentially. My diligence at OpenTable was I looked at San Francisco, which was their first market, and sales rep productivity grew over time, restaurant churn decreased over time, the number of diners per restaurant increased over time, the percentage that went that booked through OpenTable versus the restaurant’s own website moved towards OpenTable dramatically. Every metric improved. And so, you know, that’s where it both drives good engagement, but also it just improves the investment thesis.

      Sonal: The value overall, right?

      Andrew: One of the interesting points about network effects is that we often talk about it as if it’s a binary thing.

      Sonal: Right. Or homogenous, like all network effects are equal when they’re not.

      Andrew: Exactly right. When you look at the data, what you really figure out is that network effect is actually like a curve, and it’s not like a binary yes/no kind of thing. So for example, [turns to Jeff] I would guess that if you plotted the number, if you took a bunch of cities, every city was a data point, and you graphed on one side the number of restaurants in the city versus the conversion rate for that city, you would quickly find that when cities have more restaurants, the conversion rate is higher, right? I’m just guessing.

      Jeff: It’s actually almost perfect but with one refinement. The number of restaurants you have as a percent of that market’s restaurant universe; because having 100 restaurants in Des Moines is different than having 100 restaurants in Manhattan.

      Andrew: Makes total sense. So not only that, what you then quickly figure out is that there’s some kind of a diminishing effect to these things often in many cases. So for example, in rideshare, if you are gonna get a car called 15 minutes versus 10 minutes, that’s very meaningful. But if it’s five minutes versus two minutes, your conversion rate doesn’t actually go up.

      If you can express your network effect in this kind of a manner, then what you can start to show is, okay, yeah, we have a couple new investment markets that maybe don’t have as many restaurants or don’t have as many cars but if we put money into them and invest in them and build the right products, etc. then you can grow.

      You can do this kind of same analysis whether you’re talking about YouTube channels and the number of subscribers you might have, having more videos is better; I’m sure you can show that. If you go into the workplace, and you start thinking about collaboration tools, then what you ought to see is that as more people use your chat platform or your collaborative document editing platform, then the engagement on that is gonna be higher. You should be able to show that in the data by comparing a whole bunch of different teams.

      Engagement vs. retention

      Sonal: Okay… So we’ve talked about engagement and also how it applies to network effects. Are engagement and retention the same thing? I mean, in all honesty, they sound like they would be the same thing.

      Jeff: There’s overlap, but they’re different.

      Andrew: Yeah, there’s overlap, right. Just to give a couple examples: So weather is low frequency but high retention because you’re actually gonna need to know what the weather is… <Oh right!>

      Sonal: Only once a day, unless you live in San Francisco and you gotta check it, like, 20 times a day with all the microclimates.

      Andrew: Right, exactly.

      Jeff: Or if you live down here, you have to check it twice a year.

      Sonal: That’s true, it’s actually the same year-round.

      Andrew: That’s actually what it showed, was actually more that generally people didn’t really check it that often. However, you are highly likely to have it installed and running after 90 days because it’s a reference thing. You might need it.

      Sonal: It’s so important, yeah.

      Andrew: Like a calculator. Versus if you look at something like games or ebooks or those kinds of products, like, really high engagement because you’re like, “All right. I’m gonna get to…I’m gonna finish this like trashy science-fiction novel that I’ve been reading. I’m just gonna get to it.” But then as soon as you’re done, you’re like, “Okay, there’s no reason why I would go back and read it again.”

      Sonal: So the real difference is that engagement obviously varies depending on the product, the type of thing it is, whether it’s weather or ebook, and retention is are you still using it after X amount of time.

      Jeff: And different companies have different cadences. If the average user is twice a year, retention is did they book annually. Other businesses are, did they come daily? The model behind retention is completely different and the model behind engagement is completely different.

      Andrew: The chart that I’d love to really see is one that was like a bunch of different categories that’s, you know, retention versus frequency versus monetization. I think you got to be, like, really good at least on one of those axes.

      Sonal: So we’ve done sort of this taxonomy of metrics. We’ve talked about the acquisition metrics. We’ve talked about some engagement metrics, primarily frequency.

      Jeff: On engagement, it’s also time. Not just how frequent someone is, but just how much time did they spend.

      Sonal: Right. Time spent on site, on the… piece, writing comments, not just pageviews.

      Jeff: Because, I mean, the number of businesses that have great engagement is not high. Because there are only so many minutes in the day. And so, you’re just looking for where, okay, they’re just passing time and enjoying, and they both have obvious monetization associated with that behavior.

      Sonal: This is why Netflix is so freaking genius because when they literally invented the format of binge-watching, which you could not do — I love it because it’s a very internet native concept — I mean, they’ve literally fucked up everyone else’s engagement numbers.

      Andrew: I think that’s one of the narratives on why building consumer products is much harder these days. Because–

      Sonal: –And, do you think it’s true or not?

      Andrew: Well, because it used to be. It used to be that you were…what kind of time were you competing for in the first couple years of the smartphone? You were competing against literally I’m gonna stare at the back of this person’s head, or I can like use some cool app that I downloaded, right? Versus these days you actually have to take minutes away from other products.

      Sonal: Yes.

      Jeff: And it’s typically other [inaudible] because the top apps are almost all done by Facebook, Amazon, Google. And you know, breaking through just — Marc calls it the first page, the people who are on the first screen — are just such the incumbents. And sure, most people have Facebook on the screen and YouTube on the screen and Amazon on the screen.

      Sonal: It’s hard to take that down, right?

      Jeff: You have that competition. It is a big overhang right now in consumer investing because you have to displace someone’s minutes.

      Sonal: Yeah. I would add one more layer to that, at least on the content side, which is I think a lot of people make a lot of category errors because they think they’re competing with like-minded players and, in fact, when it comes to things like content and attention, you’re competing with just about anything that grabs your attention. It’s not just other media outlets. It’s…

      Andrew: …Tinder.

      Sonal: It’s a dating app. It’s something else.

      Jeff: I’m riding in the train for an hour, I could, you know, see what my friends are doing on Facebook, watch videos on YouTube.

      Sonal: It actually changes with time blocks. Xerox PARC did a really interesting study on “micro-waiting moments” and they’re literally the little snatches of time, like two seconds here and there, that you might be waiting in line or doing something, so you can do a lot of snack-sized things in that period, which is also another interesting thing to think about for how people engage with various things.

      Jeff: So it’s actually funny because there’s some businesses that have good engagement where it’s one session that goes on for a while, YouTube or Netflix or something like that. There are others that are multiple small sections that in aggregate…

      Sonal: …Like a podcast which might not finish in one sitting.

      Jeff: …Because it’s the micro-opportunities…

      Andrew: …And Google is the best example of this, right? In fact if you spend a lot of time on Google.com, you know, refining your searches and clicking around, that means actually the service is doing poorly.

      Jeff: They’ve failed. Their goal is to get you to their advertisers as fast as they can.

      Andrew: That’s a frequency play and a monetization play ultimately as opposed to an engagement one.

      Sonal: Yes, that’s fascinating.

      Andrew: And some products are more on the engagement side.

      Measuring retention

      Sonal: So sometimes you have to optimize it based on how you’re monetizing. What are some of the metrics for retention? I mean, is it just should-I-stay-or-should-I-go? Is that the retention metric?

      Andrew: I think the big thing is the concept of churn. Is a tricky one in some cases like subscription Hulu, Netflix, and then also in the SaaS world. Whether or not you’re still continuing to pay or not. And that’s really obvious.

      The thing that’s tricky for a lot of these consumer products especially episodic ones — and, it’s actually less whether they’ve quote-unquote churned or not — it’s actually just whether or not they’re active or inactive, and whether or not that’s happening at a rate that you in your business strategy have decided is acceptable or not. If every Halloween, you know how there’s those costume stores that open all over the place. If every Halloween, you go back and you buy a costume, but you’re inactive the rest of the time, have you churned or not? It’s not clear and I would argue you’ve not churned because you’re doing exactly what they want, which is to buy a costume every Halloween.

      Sonal: It seems like it smakes assessing the retention of a consumer business very difficult.

      Jeff: You adjust the time period that you’re relevant on. If the average diner dines twice a year…

      Sonal: …Then that’s your time frame.

      Jeff: You can apply that metric. Travel’s a similar thing. Airbnb is for the average user relatively infrequent. You have to tailor your look to what are they trying to do, so if you’re trying to stake up with your friends and you’re doing it twice a year, yeah, that doesn’t work. So Facebook has got a whole different setup.

      Andrew: One of the things that companies can often do is to measure upstream signal. So for example, Zillow, you’re probably not gonna buy a house very often. Maybe a couple times in your life. However, what’s really interesting is they can say, “Well, you know, maybe folks aren’t buying houses but at least are we top of mind? Are they checking the houses that are going on sale in their neighborhood? Are they opening up the emails? Are they doing searches?” Right?

      Sonal: Interesting. Why do you call that “upstream”?

      Andrew: In the funnel. You’re kind of going up in the funnel and you’re tracking those metrics.

      Sonal: I get it now!

      Andrew: As opposed to, you know, purchases. So even for OpenTable, it’s like, okay, great. Well, maybe if you’re not actually completing the reservations, are you at least checking the app for availability?

      Jeff: Or what’s new restaurants where I want to dine? There’s some level of content consumption.

      Sonal: So throughout this entire episode, there seems to be this interesting “dance” between architecting and discovering. Like, you might know some things upfront because you’re trying to be intentional and build these things, and then there are things that you discover along the way as your product and your views and your data evolves. How do you advise people to sort of navigate that dance?

      Jeff: You iterate. You develop hypotheses. You put it out there and you test the hypothesis. I think my product’s gonna behave this way. And then, did it?

      Probably the most important thing is for me, marketing can be art, marketing could be science; in the consumer internet, it’s more science. Some companies can effectively do TV campaigns, large media budgets, things like that. For me, the better companies typically just rip apart their metrics, understand the dynamics of their business, and then figure out ways to improve the business through that knowledge. And that knowledge can feed back into new product executions or new marketing strategies or new something. It’s constant iteration but it’s informed by the data at a level that on the best companies is really, really deep.

      Andrew: Ultimately, you have a set of strategies that you’re trying to pursue and you pick the metrics to validate that you’re on the right track, right? And a lot of what we’ve talked about today has really been the idea that actually there’s a lot of “nature versus nurture” kind of parts to this. Your product could just be low cadence but high monetization, and so you shouldn’t look at, you know, DAU/MAU. And so you have to find really the right set of metrics that show that you’re providing value to your customers first and foremost and then really build your team and your product roadmap and everything in order to reinforce that.

      Find the loops and the networks that exist within your product because those are the things that are gonna keeps your engagement improving over time even in the face of competition.

      Jeff: Growth is good. Growth and engagement is really really, really good.

      Sonal: That’s fabulous. Well, thank you, guys, for joining the a16z Podcast.

      • Andrew Chen is a GP at a16z in consumer technology. Prior to that, he led Rider Growth at Uber. He has written for a decade on metrics & growth on his blog/newsletter, and is the author of The Cold Start Problem.

      • Jeff Jordan is a managing partner at Andreessen Horowitz. He was previously CEO and then executive chairman of OpenTable.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Basics of Growth — User Acquisition

      Sonal Chokshi, Andrew Chen, and Jeff Jordan

      Growth is one of the most top of mind questions for entrepreneurs building startups of all kinds (and especially consumer ones) — but how does one go beyond a mindset of “growth hacking” to thinking about growth more systemically and holistically? What are the key metrics to know; why; and how?

      This episode of the a16z Podcast — one of two in a series — focuses on the user acquisition aspect of growth, followed by engagement and retention in the next episode. Featuring a16z general partners Andrew Chen and Jeff Jordan, in conversation with Sonal Chokshi, the discussion also covers the nuances of paid vs. organic marketing (and the perils of blended CAC); the role of network effects; where does customer lifetime value (LTV) come in; and much more. Because at the end of the day, businesses don’t grow themselves…

      Transcript:

      Hi everyone welcome to the a16z Podcast, I’m Sonal. Today’s episode is all about growth, one of the most top of mind questions for entrepreneurs — of all kinds of startups, and especially for consumer ones. So joining to have this conversation, we have a16z general partners Andrew Chen and Jeff Jordan. And we cover everything from the basics of growth and defining key metrics to know, to the nuances of paid vs organic marketing and the role of network effects and more. 

      Part one of this conversation focuses specifically on the aspect of user acquisition for growth, and then we cut off and go into the aspects of growth for user engagement and retention, in the next episode. But first, we begin by going beyond the concept of growth hacks — and beginning with the fundamental premise that businesses do not grow themselves…

      Sonal: So the topic we wanted to talk about today is growth, which is a big topic. What would you say are the biggest myths and misconceptions that entrepreneurs have about growth?

      Andrew: You know, not only is there the misconception that it happens magically, then the next layer I think is that it’s really just like, oh, a series of, you know, tips and tricks and growth hacks that kind of keep things going as opposed to like a really rigorous understanding of, you know, how to think about growth not just, as kind of the top line thing but actually that there’s acquisition, that there’s engagement, that there’s retention, and each one of those pieces is very different than the other and you have to like tackle them systematically.

      Jeff: It is a scientific discipline, done right, because it requires you to understand your business and business dynamics at this incredibly micro level.

      Sonal: I love that you said that because one of the complaints I’ve heard about “growth hacking” is that it’s just marketing by a different name, and what I’m really hearing you guys say is that there’s a systemic point of view, there’s rigor to it, there’s stages, there’s a program you build out.

      Jeff: If you’re fortunate enough to achieve product-market fit and your business starts to take off, typically, you know, when in the wonderful situation do you get this hyper growth where you’ll grow year over year, you know, it’s triple digits. It’s just exploding. And then gradually the law of the large numbers starts to kick in and maybe the 100% growth becomes 50% growth the next year, and then the law of large numbers continue to kick in and there’s 25% and then it’s 12.5% and so growth tends to decay over time even in the best businesses. And so the–

      Sonal: — Didn’t you use to call it like “gravity”?

      Jeff: I called it gravity, you just would…it comes down to earth. And then the job of the entrepreneur is to be looking years down the road and say, “Okay, at some point growth in business A is going to stop and so I want to keep it going as long as I can and there’s a whole bunch of tactics to do that,” but then the other tactic, the other strategies, okay, I need new layers on the cake of growth.

      At eBay the original business was an auction business in the U.S. and so, you know, some of the things we layered on early days we layered on fixed price in the U.S. — it’s not revolutionary but it really did grow then we went international. And then we layered in payment integration and each time we did that the total growth of the company would actually accelerate which is very hard to do at scale.

      Sonal: That’s the whole point… like there’s intentionality to it. It’s not an accident that you guys introduce new businesses, new layers on the cake.

      Jeff: Businesses don’t grow themselves, the entrepreneur has to grow them. And, you know, occasionally, you stumble into a business that seems to almost grow itself but they’re just aren’t many of those in the world and that growth almost never persists for long periods of time unless the entrepreneur can figure out how to continue its growth.

      Sonal: Right. I remember a post you wrote actually a few years ago on “The ‘Oh, Shit Moment!’ When Growth Stops” because people are a little blindsided by it.

      Jeff: And that’s the flip side of it. You know, early on you get this great growth, you had to keep it going. When it stops your strategic options had been constrained dramatically.

      Andrew: A lot of times when you’re looking at what seemingly is an exponential growth curve. In fact, it’s really something like, oh, you’re opening in a bunch of new markets, right, so there’s sort of a linear line there, but then you’re also introducing products at the same time and you’re also reducing friction and, you know, sign-ups or retention or whatever, and so, the whole combination of those things is really kind of like a whole series of accelerating pieces that looks like it’s, you know, this amazing viral growth curve. But it’s actually like so much work underneath.  <Sonal: Right.> You know, that makes that happen.

      Sonal: I’ve also heard you [Andrew] talk about, being able to distinguish what is specifically driving that growth, so you don’t have this like sort of exponential-looking curve without knowing what that lever that you’re pulling to make that happen or knowing what’s happening even if it’s kind of happening naturally or organically. Can we break down some of the key metrics that are often used in these discussions including just what the definitions are and maybe just talk through how to think about them?

      Andrew: Right. Yeah, so when you look at a large aggregate number like, you know, total monthly active users, right, or you’re looking at like —

      Sonal: — “MAUs”

      Andrew: –Yeah, MAUs, right. Or you’re looking at, you know, the GMV like all the…adding up all the transactions in your marketplace–

      Sonal: — So, “gross merchandise value”.

      Andrew: Yup. And so, you know, when you look at something like that and if it’s going up or down, you don’t have the levers at that level to really understand like what’s really going on. You want to go a couple levels even deeper: How many new customers are you adding? As you’re growing more and more new customers, a bunch of things happen. If you’re using paid advertisement channels, things tend to get more expensive over time because — you know, your initially super, super excited core demographic of customers — like they’re gonna convert the best and as you start reaching into different geographies, different kinds of demos, all of a sudden they’re not gonna convert as well, right?

      Sonal: Just to pause in that for a quick moment, you’re basically arguing that growth itself halts growth in that context.

      Andrew: Right. Yeah. So the law of large numbers means that you know there’s only a fixed number of humans on the planet, there’s only a fixed number of people that are in your core demographic, right? Once you surpass a certain point, it’s not like it’s it falls off a cliff, it’s just more gradual that you know that the customer behavior really changes.

      Sonal: How do you determine what’s what when you don’t have product-market fit? Sometimes aren’t these metrics ways to figure that out or is this all when you have product-market fit… like is there a pre- and a post- difference between these?

      Andrew: Very concretely, you want to understand how much of the acquisition is coming from purely organic (people discovering it, people talking to each other), as opposed to, oftentimes you’ll run into the companies that have over 50% of their acquisition coming from paid marketing and that tells you something that you’re, you know, needing to spend that much money to get people in the door.

      Sonal: Yeah. So CAC, “customer acquisition cost”, that’s what you’re talking about when you talk about acquisition.

      Jeff: CAC is what it cost to acquire a user, “blended CAC” is what it costs to acquire a user on a paid basis plus then also what free users you acquire. So if you’re acquiring half your users through paid marketing you’re paying a $100 to acquire a user but half of your users are coming at zero, paid CAC is 100, blended CAC is 50.

      I think blended time is a really dangerous number. Most of the best businesses in the internet age of technology haven’t spent a ton on paid acquisition. And so the truly magical businesses, you know, a lot of them aren’t buying tons of users… Amazon’s key marketing right now is free shipping. And then, yeah, the economics of paid acquisition tend to degrade overtime.

      Sonal: As it grows.

      Jeff: As it grows and you just try to scale it and, you know, largely you’re cherrypicking the best users and then you’re trying to also scale the number you get to grow. I need twice as many new users this year as last year and you typically pay more so that magical LTV to CAC ratio which early on says, “Oh, we are three to one, you know, in two years it’ll probably be one and a half to one if you’re lucky,” or something like that. So we typically do try to look for these other sources of acquisition be it viral, be it, you know, some other form of non paid <crosstalk>

      Sonal: I want to quickly define LTV — it’s “lifetime value” of the customer, but what does that mean?

      Jeff: When you’re showing an LTV to CAC ratio you have no idea of what you’re seeing essentially given all the potential variations of the numbers. So we will almost always go for clarity. LTV, lifetime value, should be the profits, the contribution from that user after all direct costs.

      Sonal: How do we define the LTV to CAC ratio? What do the two of them in conjunction mean?

      Jeff: Well, let’s break them down. LTV is lifetime value. What you’re describing there is the incremental profit contribution for a user over the projected life of that user. So not revenue per CAC is that you know typically there’s cost associated to user. What’s the incremental contribution that the user brought from that <crosstalk> <Sonal: And that you mean the user brought to your company’s value.> To the company, yeah.

      Sonal: So it’s a value of your customer to the bottom line?

      Jeff: It’s the value of each customer to the bottom line, and then you compare that to the CAC or “cost of acquired customer” to understand the leverage you have between what I need to spend to acquire a customer and how much they’re worth. If your CAC is higher than your LTV you’re sunk. Because it’s costing you more to acquire a user…

      Sonal: Than the value you get out of it. Now I get it.

      Jeff: …then you’re going to get out of that user.

      Sonal: Yeah.

      Jeff: If it’s the opposite, at least you’re in the game. You know, I get more profit out of the user than I get the cost to acquire that user. And then there’s this dynamics on how does it scale over time, CAC tends to go up, LTV tends to go down. Because you’re, on the CAC side, you’re acquiring the less interested users over time. So they cost more to acquire and they’re worth less, and so that the LTV to CAC ratio, in our experience, almost always degrades as over time with scale.

      And so, you know, when you’re in that conversation, you’re in a very specific conversation of, “Okay, how much room do you have?” “How is it gonna scale?” “You know, what’s gonna impact your CAC like a competitive thing?” So there has to be a lot, it had to be like 10 to kind of get you over that concern that oh, my goodness, those two were so close, that you have no margin for error.

      Sonal: Right. This also goes back to the big picture, the layers on the cake, because if you have other layers you don’t have to only worry about one layer CAC to LTV ratio.

      Jeff: It really does affect the calculation. If it’s, I’m in a new business, and I have a whole different CAC versus, you know, LTV ratio then that’s a different conversation as well.

      Sonal: And the big picture there, is that if you don’t know the difference of what’s doing what when you may get very mistaken signals, mixed signals about your business, and so you guys don’t want blended CAC because you want to know what’s driving the growth.

      Andrew: I think what blended CAC gives you is it gives you a sense for at this particular moment in time, you know, what’s happening. The challenge is that when it comes to paid marketing, in particular, it’s easy to just add way more budget and a scale that than it is to scale organic or to scale SEO. So your CAC is giving you a snapshot, but then as you’re trying to scale the business you’re trying to increase everything by 100% over the next, you’re trying to double everything then all of a sudden, you know, your blended CAC starts to approach whatever your dominant channel actually looks like.

      And so if you’re spending a bunch of money then it’ll just approach whatever is your paid marketing, you know, CAC. What entrepreneurs should think about is what is the unique organic new thing that’s gonna get it in front of people, without spending a bunch of money, right?

      Jeff: A lot of the best businesses have this very interesting, I’ll call it a growth hack. I mean OpenTable, when I was managing it, did not pay any money at all to acquire consumers. Like how can you do that? You know, it had millions of consumers. The restaurants would mark it OpenTable on our behalf.

      Sonal: Right.

      Jeff: They go to The Slanted Door website like when they were an OpenTable customer and you’d see, you’re looking…you go there to try to get the phone number to make a reservation and they’d say, “Oh, make an online reservation.” And we then got paid to acquire that user in its core form. But that hack was a wonderful thing. It scaled with the business and got us tons of free users.

      Sonal: To be fair, and this is another definition we should tease apart really quickly before we move on to more metrics, that also had a quality of network effects which we’ve talked a lot about in terms of these things growing more valuable to more people that use it… is that growth? What’s the difference there?

      Jeff: Well, the business grew into the network effect. The key tactic to build the network effect was that free acquisition of consumers that the more restaurants we had, the more attractive it was to consumers the more consumers who came, the more attractive it was to restaurants. So there is a wicked network effect.

      Sonal: Like a flywheel effect, right.

      Jeff: If you’re not spending anything on paid acquisition of consumers, how do you start it? And the placements that OpenTable got in the restaurant book both physically in the restaurant but particularly in the restaurant’s website was the key engine that got the network effect started. You had to manually sell some restaurants come for the tools, stay for the network, but then once the consumers got enough of a selection and started to use it, it was game over.

      Sonal: Right, that was one way of going around the bootstrapping or the chicken-egg problem and seeding a network.

      Andrew: Network effects have…there’s a lot of really positive things about them and one of the big pieces is that virality is a form of like something that you get with the network. You know, the larger your network is, the more surface area, the more opportunities you have in order to encounter it, right. And so, you know, in the case of Uber (where I was recently), by seeing all the cars with the Uber logo like those are all opportunities to be like, “Oh, what is this app? I should try it out.” And so it’s mutually reinforcing: then you get more riders and then you get more drivers that are into it and so, I think all of that kind of plays together.

      Jeff: I’ll bring two examples up, the pink Lyft mustache when I first got to San Francisco.

      Sonal: I remember that.

      Jeff: You can see it once in the car and you’d go, “Oh, that’s pretty weird.” You see it twice in the car and you say, “Something is going on here that I don’t know about, and I have to understand what it is.” Lime is the same kind of thing.

      Sonal: Right.

      Jeff: They’re bright green and they glow essentially. So when someone sees one in the wild, someone bolts by them in a glowing green electric scooter and you’re just like, “Okay…what is that?” And Lime hasn’t spent a penny on consumer acquisition. <Sonal: Right.> Because their model is such that physical cue in the real world leads to it.

      Andrew: The other one I’ll throw in as well is within workplace enterprise products there’s a lot of kind of bottoms-up virality that comes out of people, you know, kind of sharing and collaborating.

      Sonal: Like with Slack.

      Andrew: Yeah, like for example Slack is a great, it’s an example of this. And so, these are all kind of really unique ways that you can, you know, get acquisition for free. And so then your CAC is, you know, “zero” as a result.

      Sonal: Yeah. You guys have talked a lot, about organic. It makes it sound to me as a layperson that you don’t want paid marketing! Like what’s your views on this — is it a bad thing, is it a good thing; I don’t mean to moralize it but — help me unpack more where it’s helpful and where it’s not. Are they any rules-of-thumb to use there?

      Jeff: I mean a lot of great businesses that have leveraged paid marketing. The OTA sites (online travel agencies – Priceline and Expedia) just spends, you know, they spend a GDP of many large countries in their acquisition; and then it’s often a tactic in some good business. But if it’s your primary engine, a couple of things happen: One is it tends…the acquisition economics tend to degrade over time for the reason we’re saying…  <Sonal: Right this is…> And it leaves you wide open to competition.

      Sonal: It gets commoditized basically.

      Jeff: If you need to buy users, I mean if you’re selling, you know, the new breed of mattress and you need to buy users and early on, you’re the only person competing for that word, flas-hforward a year or two, they’re like six new age mattress manufacturers with virtually identical products competing for the same consumer. The economics are not going to persist over time. And so, you know, one of the key questions in businesses driven by heavy user acquisition is how does the play end? You know, it usually looks pretty good at the beginning of the play but in the middle it’s starts getting a little complex and there’s tragedies at the ends.

      Sonal: There’s literally an arc.

      Andrew: And I think, you know, if it is something that you’re using in conjunction with a bunch of other channels and you’re kind of accelerating things, that can be great. For example when Facebook in the past broke into new markets they started with paid marketing to get it going. And so in a case like that really paid marketing is a tactic to kind of get a network affect jumpstarted right? <Sonal: Gotcha.> And then you can kind of like pull off from that if you’d like. <Sonal: Right.>

      Andrew: But if you’re super, super dependent on it and you don’t have a plan for a world that you know all the channels atregonna degrade [in] then you’re gonna be in a tough spot in a couple of years.

      Sonal: Totally. Do you have sort of a heuristic for when to stop the paid? Is there like a tipping point, you know, THIS is when you move?

      Andrew: I think in terms of how much paid should you do as part of your portfolio, I think that’s the right way to think of it is it’s one out of a bunch of different channels, right? And so I would argue the following: First is you really have to measure the CAC and the LTV and be super disciplined about not spending ahead of where you want it to be and not to do it on some, you know, blended number that doesn’t make any sense. <Sonal: Right.> And then I think the other part is you really want it to be kind of a small enough minority of your channels. Such that if you were to get to a point where it turns out to be capped that you’re okay, that you can live with that.

      Sonal: Your business will survive and you continue to grow and be healthy.

      Andrew: Right, exactly, and you can still get the growth rates you want and you can still, you have such strong product-market fit that you’re able to maintain that.

      Jeff: Take a couple of sector examples. You know, ecommerce, a lot of companies struggle with, “Okay, how do I get organic ecommerce traffic?” So most ecommerce companies rely heavily on paid user acquisition, you know, typically one of the interesting things is they degrade over time and they’re all competing for the same user. It’s hard for ecommerce companies in most segments to be profitable and you’d look at the same kinds of dynamic and restaurants delivery. You know, if you can’t differentiate yourself and you’re highly reliant on paid marketing, the movie typically doesn’t end really great, and so, we look for segments where there’s a balance or they come up with that really unique growth hack and they’re not then reliant on paid channels.

      And then by the way, paid channels can degrade too. I mean, I’d made a couple of investment mistakes where the paid acquisitions looked really good and then actually what they were doing is they’re arbitraging something like Facebook’s early mobile attempts where the people who participated with Facebook mobile ads early got real deals. They were nowhere near kind of the price they should have been trending at, so you’re like, “Look at these user analytics. They’re awesome!” And then Facebook, you know, kind of got the equilibrium when supply and demand met and the cost went up multiples, and those businesses that looked so good early just got incredibly stressed because they had no alternative to that inflation.

      Sonal: That’s the case of platform risk where you’re dependent on the channel of on Facebook mobile or whatever the specific channel was there. But Andrew, you were also earlier talking about just a cap on how much is possible, and you both referenced the fact that things can become very competitive, that your competitors can also buy the same channels and then it gets very crowded or very expensive. So there’s multiple layers of the risk of the paid is what I’m hearing, but you have to be aware of that.

      Andrew: Yeah. So I think on the acquisition side today, there’s a couple of really interesting opportunities that might be, you know, temporal, right, and like it may go away, right? <Sonal: Like, anything that crosstalk> For example, I think that if you have a product that is very highly visual, and I think this is, you know, one of the reasons why eSports has gotten so huge is because you have a product that naturally generates a ton of video in an age which all the platforms are trying to rush to video.

      Sonal: That’s fascinating.

      Andrew: Right? And so, you know, maybe this will be less of an opportunity coming up but like, you know, that’s a thing.

      Sonal: Why would you say that’s temporal because it seems like…

      Andrew: Because the competition will…

      Sonal: …Do the same thing?

      Andrew: …Yeah, will do the same thing, right. I think we’re now gonna move to a thing where all of these different kind of software experiences all are incredibly sharable. Like there’s no point these days in building a new game that doesn’t have like built-in recording and publishing the Twitch stream. And built-in tournament systems and all the community features and all that stuff that you need and, you know, I think it used to be that you would think of a game is just the actual IP but in fact, it’s sort of these layers and layers and like social interaction and content around it. And I think that’s about true as well as, all of these different brick-and-mortar experiences that are making themselves highly Instagrammable, they are adding the areas where you actually stand there and pose…

      Sonal: Oh, my god, my favorite story about this is the restaurant trend of making square plates and layouts so it really fit beautifully with Instagram. That’s like one of my favorite cases; that’s one of my favorite things in the world is when the physical world adapts to the digital!

      Andrew: And then you can go the other way too which is, physical products like scooters that remind you to engage digitally. The other, fun example I always like is everyone’s had the experience now where they’re just like in the room talking and then their Amazon Echo just turns on and it’s trying to go and I’m like, you know, they have no incentive to fix that. <Sonal: Yeah.> Because it reminds you that it’s there and reminds you to talk to it.

      I think the big takeaway here is that you have to really be creative and really be on the edge of what everyone’s doing, right? And so if it turns out that everyone’s really into video and they’re really into Instagram right now, you have to think about like how does my product actually fit into that trend? <Sonal: Yeah.> And if you can find it, then you can get an amazing killer way to get jumpstarted and if the trend lasts then great, accelerate it with paid marketing, accelerate it with PR, do all that stuff to kind of keep it going.

      I also want to make the distinction that we’re mostly talking about growth and acquisition.

      Sonal: Yesss!

      Andrew: And that is what startups mostly care about in the early days, because you don’t really have any active users, right? But the other part of this is that you see all the users would show up and how active they are starts to change over time… <overlap/crosstalk>

      Sonal: <overlap/crosstalk>…The engagement. Well, thank you guys for joining the a16z Podcast.

      [see part two on growth for user engagement and retention]

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Andrew Chen is a GP at a16z in consumer technology. Prior to that, he led Rider Growth at Uber. He has written for a decade on metrics & growth on his blog/newsletter, and is the author of The Cold Start Problem.

      • Jeff Jordan is a managing partner at Andreessen Horowitz. He was previously CEO and then executive chairman of OpenTable.

      Network Effects, Origin Stories, and the Evolution of Tech

      W. Brian Arthur, Marc Andreessen, and Sonal Chokshi

      “The rules of the game are different in tech,” argues — and has long argued, despite his views not being accepted at first — W. Brian Arthur, technologist-turned-economist who first truly described the phenomenon of “positive feedbacks” in the economy or “increasing returns” (vs. diminishing returns) in the new world of business… a.k.a. network effects. A longtime observer of Silicon Valley and the tech industry, he’s seen how a few early entrepreneurs first got it, fewer investors embrace it, entire companies be built around it, and still yet others miss it… even today.

      If an inferior product/technology/way of doing things can sometimes “lock in” the market, does that make network effects more about luck, or strategy? It’s not really locked in though, since over and over again the next big thing comes along. So what does that mean for companies and industries that want to make the new technology shift? And where does competitive advantage even come from when everyone has access to the same building blocks (open source, APIs, etc.) of innovation? Because Arthur — former Stanford professor, visiting researcher at PARC, and external professor at Santa Fe Institute who is also known as one of the fathers of complexity theory in economics — has written about the nature of technology and how it evolves, observing that new technology doesn’t come out of nowhere, but instead, is the result of “combinatorial” innovation. Does this then mean there’s no such thing as a dramatic breakthrough?!

      In this hour-long episode of the a16z Podcast, we (Sonal Chokshi with Marc Andreessen) explore many of these questions with Arthur. His answers take us from “the halls of production” to the “casino of technology”; from the “prehistory” to the history of tech; from the invisible underground autonomy economy to the “internet of conversations”; from externally available information to externalized intelligence; and finally, from Silicon Valley to Singapore to China to India and back to Silicon Valley again. Who’s going to win; what are the chances of winning? We don’t know, because it’s a very different game… Do you still want to play?

      Show Notes

      • How the concept of network effects was born [1:05], its initial reception, and eventual adoption [9:53]
      • Discussion of diminishing returns [18:09] and the role of timing or luck [22:16]
      • How new technologies are created from existing elements [25:12]
      • The impact of computer networks [38:44], and the societal impact of machine intelligence [46:42]
      • Globalization and economies around the world [53:52]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. So, today we have a special episode. We talk a lot about network effects as one of the most important dynamics, especially in software-based businesses. You can see much of ours and others’ thinking on the topic at a16z.com/networkeffects. But today, our special guest is W. Brian Arthur. He’s widely credited for first describing network effects, and beyond that, has had a long and very influential career in economics, especially as applied to the tech industry. So I asked Marc Andreessen to co-host and add a little color commentary. But first, more about Brian.

      Brian was formerly a professor of economics at Stanford, is a visiting researcher at PARC, formerly Xerox PARC, and is also an external professor at the Santa Fe Institute — because besides his foundational work in network effects, he’s also considered one of the fathers of complexity theory — has written books on the nature of technology and how it evolves, and has also written a number of pieces on AI and the autonomy economy, all of which we’ll touch upon in today’s episode. We also cover a lot of neat history in between, and we end on the topic of innovation clusters around the world, including Silicon Valley. But first, we begin briefly with where Brian’s ideas came from.

      The concept of network effects

      You’re a really influential economist who’s — and I sometimes make fun of economics.

      Prof. Arthur: Feel free.

      Sonal: I know. But, you know, your work has really actually driven so much, or described so much, of what actually happens in technology, and there seems to be a gap often between the worlds of economics and technology, and you’re really at the heart of that. So why don’t we start with some of your most seminal work, starting with your famous classic paper around increasing returns and positive feedbacks.

      Prof. Arthur: Sure.

      Sonal: Like, if you were to just sort of distill and summarize some of the key concepts and how it contributed to the tech industry.

      Prof. Arthur: Sure. To go back a little bit, I’ve been interested in technology for a long time. I was trained as an engineer, and then mathematician and operations researcher — basically algorithmic theory. So, my basis is actually technology, and then I added as a layer on top of that — I fell into the wrong company and became an economist. And I arrived in Stanford in 1982. At that time, Silicon Valley was blossoming. We said in ’82, it was all about electronics, then it was about computation, then the web, then the cloud. Now it’s about AI. So, Silicon Valley keeps morphing and changing. I was enormously taken, just by the sheer energy of the place in the early ’80s, and on through 26 years or so since. It keeps recreating itself. It’s like looking into, I don’t know, it’s like looking into some cauldron of everything bubbling and changing all the time. And it became very clear to me that there was a phenomenon going on in technology that you didn’t see so much in the rest of the economy.

      Sonal: Right. The phenomenon of network effects, which I should clarify in your papers is also named positive feedbacks, and increasing returns.

      Prof. Arthur: Yeah. In standard economics, if you get very large in the market, everybody runs into some sort of diminishing returns, and markets tend to balance. The market’s fairly well-shared. That was the theory when I came along, but it didn’t seem to me that tech worked that way. Go back to about 1982, ’84. At the time, we had VHS and we had Beta. Those were the basic operating systems for video recorders, and one of them happened to be better — Betamax was better — and I started to wonder why VHS dominated the market.

      Sonal: I’ve always wondered this, actually.

      Prof. Arthur: And then I realized that a host of small events early on had pushed VHS into a slight lead, and if you were going down to your local movie rental store — again, this is…

      Sonal: Back when Blockbuster existed.

      Prof. Arthur: …Blockbuster, you would tend to see more VHS movies. That meant you’d get a VHS recorder, and that meant that they would stock more VHS.

      Sonal: Aren’t those complements, in economic terms?

      Prof. Arthur: Oh, yeah. The two were kind of interacting. The more VHS is out there, the more I buy VHS. So I began to realize that I was seeing this in market after market. There weren’t diminishing returns. If VHS got ahead, it would get further advantage. The whole thing was quite unstable, and small events tilted you towards Beta or VHS. My analogy was, this was like bowling a ball perfectly down the middle of an infinitely long bowling alley. It could stay quite long in the middle, but if it started to drift to one side, it would go further, and then it would fall into the gutter at the side, and that side would lock in the market, so to speak.

      Sonal: And by the way, you borrowed — I think I remember you telling me that you borrowed the lock-in jargon from the military, like locking in on a target.

      Prof. Arthur: Yeah, the — lock-in wasn’t used heavily at the time. I’m sure there are other people who used the phrase, but with fighter jet radar, when you’re going at very high speed and you’re pursuing an enemy or something, or maybe a radar station itself on the ground, you lock into the target. It’s not just that you find the target, but you want to lock on to that target, and then you can release your weapons, and the weapons will stay locked into that particular…

      Sonal: Right. I remember this from “Top Gun.” I mean, that was also very popular.

      Prof. Arthur: So I borrowed “lock in,” and since, that’s become very popular. We’re locked into this, we’re locked into that. Basically, meaning that small chance events have landed you into something you can’t get out of. So what I realized through quite a few phenomena that have become famous — since this was all very embryonic in my mind — that the sort of firms I was looking at, if one of them got ahead out of half a dozen, it could get further ahead. You couldn’t predict which one would get ahead. It would start to get enough advantage that it could dominate the market and get still further ahead. It would lock in. It would have so much cost advantage — or now we’d say it’s so much user base — that it would be hard to dislodge. Microsoft got ahead with certain contracts very early in the game. They locked in a lot of the personal software in the 1980s.

      Similarly, other systems came along since. There were search engines like AltaVista, as well as Google and others. Google gets ahead and began to dominate that market, and now has it pretty well locked in. You could say similar things for social media. So it was a general phenomenon, that anything that got ahead — because you wanted to be with the majority of people — could get further ahead. We now call it network effects. Companies like that set up a network of users, you want to be with the dominant network because your friends are with that, or…

      Sonal: It’s more valuable the more you use it — are in it.

      Prof. Arthur: Yeah. Or you know more about it, you hear more about it, or you understand it better. Five generations ago, none of our ancestors spoke English, but we’re all speaking English now.

      Sonal: English is a network effect? I’ve never thought about that.

      Prof. Arthur: We speak English because we wanna be understood by everybody else.

      Sonal: Right. You’re right. I never even thought about it.

      Prof. Arthur: And if small events had gone otherwise in the 1700s, it might’ve been French. Or, if you were betting in the 1500s, it could have been Latin or whatever.

      Sonal: So, how was it received, when you first put out this paper arguing against diminishing returns in tech — more towards positive feedbacks, increasing returns?

      Prof. Arthur: Well, I wrote a paper on this in 1983, sent it to four leading economics journals. Not all at the same time, one after another. I finally got it published 6 years later in 1989.

      Sonal: So they didn’t really accept it?

      Prof. Arthur: They did not like it. I kept getting reviews saying, “We can’t find fault in this, but this isn’t economics.” And in the meantime, the idea was out there, but there was no citation because no journal dared to publish this. There’s a good reason. In those days, what I was saying is that the economy could lock in to technologies — or to products, or even to ways of doing things — that might be inferior, because that came up, maybe, early on by chance and got locked in. And during the Cold War, in the mid-’80s, this was not popular. I gave the talk in Moscow in 1984. I was saying, in a capitalist economy, you can lock in to an inferior product. Hands went up, you know, <in a Russian accent> “Professor, we want to point out that in Soviet Union, such a thing not possible because with socialist planning, we do not make such mistakes.”

      Marc: The central planners will dictate the correct outcome.

      Prof. Arthur: Yeah. I came back to Stanford, got a Ph.D student. I said, “Figure this out. I don’t believe it.” He did. He wrote a beautiful paper. Robin Cowan is his name. And he showed that even with the best of planning, you can’t foresee what’s gonna happen…

      Sonal: That’s fascinating.

      Prof. Arthur: …and of course, you can lock in to the wrong thing. Economists hated this. The whole idea was, everybody’s free to choose, and that lands you in the right solution. And I thought, “Is that correct? I’m free to choose. Do we always choose the best spouses? Social statistics might suggest otherwise.” But what it made for was a very different game in Silicon Valley.

      Early reception and current views

      Sonal: So, speaking of it being a different game. You know, we have a lot of entrepreneurs that listen to our podcast. How does it change the game? Because people always use the phrase “game-changing” very freely.

      Prof. Arthur: Well, first of all, entrepreneurs in Silicon Valley are really smart, and they didn’t exactly get all these ideas from me. I’m not being modest, I’m just being realistic. When I brought out this theory, it kind of corroborated their intuition. So, what I’d say is this. If you are thinking in standard terms — go back to brewing beer, or a company like General Foods — if you want to make profits, you’re thinking of getting production up and running properly, getting your costs down, making sure everything’s terribly efficient. The game was different in tech. The whole game was to try to, early on, grab as much advantage as you could. And I remember that I wrote a paper on this — the “Harvard Business Review,” in 1996. And as that paper got circulated very widely in Silicon Valley, I remember hearing one story that Sun Microsystems had developed Java, and naturally, that cost a huge amount of money. So the guys with the green shades…

      Sonal: The accountants?

      Prof. Arthur: …were saying, naturally enough, we should charge a huge amount of money for anybody who buys this. And the other people, who had read this theory, said, “No, no, no, no, no. Give it away, give it away, give it away for free.” And there was a tremendous hullabaloo over this. And finally, somebody took my article and just slammed it in…

      Sonal: Fighting with papers, I love it.

      Prof. Arthur: …Scott McNealy’s desk, and it was game over. He got the point immediately that what you do in an increasing returns market is you try to build up your user base. Now, that’s become completely intuitive since. There was a time it wasn’t standard, that the accountants were saying, “We need to amortize all the R&D money, and we need to get that outlay back as fast as we can, so we’ll charge arms and legs. Later, we can drop the cost.”

      Marc: Right. It requires deferral of gratification, right? It requires long-term thinking.

      Prof. Arthur: That’s right.

      Marc: It requires, in other words, not only strategic thinking, but also long-term thinking.

      Prof. Arthur: Long-term thinking.

      Marc: You have to project forward to what the economics will be when you win.

      Prof. Arthur: Yeah. And, again, I think that that makes [a] very different atmosphere in tech. Tech is not about making profits. It’s about positioning yourself in markets, and trying to build up user base, or network advantages — trying to build on those positive feedbacks. Think of amazon.com. For years and years, they kept reinvesting and kept betting on the positive feedbacks, and eventually, they dominated that whole market. Now they can make huge profits and keep expanding. But it gives you a very different way of thinking. I called the standard way of doing things, “The Halls of Production” — you know, these big factories — but it seemed to me that what was happening in tech was not the halls of production. I called that “The Casino of Technology.”

      As if you had this huge marquee. There are many tables, with different games going on, you know — oh, yeah, we’re doing a game on face recognition over here, or whatever. And people come up to the table, just as search engines say, “Okay, who’s gonna be here?” “We don’t know. The technology hasn’t really started.” “What’s the technology gonna be like?” “I have no idea, Monsieur.” “How much will I have to put upfront?” “Well, you know, you could join the game, Monsieur for maybe 1 billion.” “What are my chances of winning?” “I have no idea. Perhaps if there are 10 players, your chances might be 1 in 10. Do you still want to play?” So it was a very different game.

      And I don’t want to make it sound like too much luck, because the particular entrepreneurs who — kind of, knowing that their technology was right — and they had a sort of instinctive idea of positioning the technology, and building that user base early. Rather than saying, “We want to get profits out of this.” The game keeps changing, but my point is that the basic game in tech is not the same as the basic game in standard production. And every once in a while, you see somebody taken from the standard production side of the economy — some CEO — brought into a tech firm, and they don’t quite get it. The classic case was Apple. The classic case was Sculley.

      Sonal: That famous quote, “Do you want to sell sugar water for the rest of your life?” For him to be enticed away from a beverage company to work at Apple?

      Prof. Arthur: Yeah. And CEOs are very smart indeed, but it’s not just a matter of intelligence. It’s a different way of thinking. And it’s so familiar to us now — this new way of thinking in the valley, in Silicon Valley — that we take it for granted that we always thought that way. But we didn’t.

      Marc: Do you think that — I don’t know if you have a view on this or not. Do you think financial markets understand this, to the degree that they should, even after all this time?

      Prof. Arthur: No. I’ll give you two instances. Warren Buffet very, very famously said, “I don’t dabble in high-tech.” He says, “I don’t touch that, simply because I don’t understand it.” A friend of mine, Bill Miller of Legg Mason — I’ve known him for 20 or more years through the Santa Fe Institute. Bill read this stuff, got it, understood it, and did extremely well. So the best answer I can give to that is, it’s not general knowledge among investors fully yet. Certainly wasn’t 20 years ago. But there’s an increasing number of people who get that the rules of the game are different in tech from in standard business.

      Marc: One of the smartest hedge fund managers I know — he says there’s still, in financial markets — is what he calls the New York-Palo Alto arbitrage, right? And basically, he said his strategy is [to] spend half his time in New York, and understand what all those assumptions are — which basically are the drivers — New York is the driver of asset prices. That’s where most of the really smart investors are, at least in the U.S. And then he says, basically, come to Palo Alto, figure out all the ways they’re wrong, and then place the contrary bet. And the theory, I think, that you’re laying out underneath that is, basically — you might say that the New York mindset stereotypically might be the “halls of production” mindset, even still.

      Prof. Arthur: Yes, that’s right. Yeah.

      Marc: Right.

      Prof. Arthur: It certainly is that way in Europe. I’m always amazed, and slightly appalled, that people think of technology in Europe as something that’s done by very big companies, and it’s pretty good technology. But they don’t get that this is a game of positioning, of building a user base. And it’s well understood in California. It gets less well-understood on the East Coast, and then not very well understood elsewhere.

      Marc: So, question. Another very smart guy, Peter Thiel, takes it a step further. He asserts that in the long run, every kind of industry — every kind of product — either becomes a monopoly or a commodity. In other words, in the long run, the margins either go to infinity, or they’re 100% — or they go to zero. And it’s just a question of time, and if you don’t have increasing returns, you’re on a long-term downward slide to commodity. <Yep.> And he asserts that the things we view as intermediate cases — businesses today that are, like, 20% margins — are fated to decline to zero over time. Is his view, do you think, too extreme, or would you support even a view, kind of, that stark?

      Prof. Arthur: I like the idea. I think he’s basically on target, but there are perennially commodity industries — I’m thinking of airlines — where the margins are pretty low. They’re usually lower than 10%, but still these persist, and quite often, governments intervene. Yeah, I have a lot of sympathy for Peter Thiel’s view. I think that in the long, long run, things do tend to get dominated by only one or two players, even in the standard businesses. And the reason that’s not completely and utterly true all the time is that there are new products getting launched all the time in standard product space. And that keeps us in this more standard economic setup.

      Diminishing returns and luck

      Sonal: When you describe the work on increasing returns, <Yeah.> you also mention the flip side of this, sort of, effect of increasing returns — which is sometimes you might get to the point where the network can go back to a point where it goes to diminishing returns.

      Prof. Arthur: Yes.

      Sonal: For example, if there’s too many listings in a marketplace, or something. Do you have any thoughts on that, or any new takeaways around that? Because if the network is more valuable as more people use it, why would there be a diminishing return at a certain point, if it gets too big? Like, is there an ideal size?

      Prof. Arthur: No, I don’t think so. I think it depends very much on the network itself. Some networks can eventually become commoditized. And so, if it’s a commodity, anybody can, sort of, come in and offer the same thing. But a much more common pattern, and the pattern that I would expect, is that there is a network. Go back to 1984. Microsoft moves in, other companies move in, Microsoft dominates. But eventually what happens in [an] increasing returns market is that the next invention comes along. And some other company [that] is offering web services, or something, comes to dominate. So you can dominate for a while in one large niche in the digital economy, but then the next set of technology comes in, and new players come in with that. Google recognizes this, and Google’s trying to stay ahead of them by being in on the new technologies.

      Sonal: Well, it reminds me [of] when they tried to do, like, social networking when Facebook came along, and now they’re, sort of, just decided to become an AI-first company. So it kind of brings that full circle.

      Prof. Arthur: Yes, that’s right. But companies don’t always make that transition from one technology to the next very well. Apple’s been very lucky, where they’ve invented some of the technologies, and then they’re able to surf on that new board, so to speak. But the overall thing is that lock-ins tend to last for a certain amount of time, and then they become obsolete, and some new game comes along.

      Sonal: Right. Or they become ubiquitous, utility-like — and the new game still comes along. Because I would argue that Google’s always gonna be around for search. <Oh yeah, sure.> Because they’ve, sort of, dominated that market, but they may become, like, utility in that application — if not in something new.

      Prof. Arthur: That’s right. And then the advertisers may drift off to something hotter.

      Sonal: You mentioned earlier that you don’t think it’s luck, and this discussion makes it almost sound like it’s an accident that there’s a winner-take-all effect. But is there some way of knowing early on — the entrepreneur who maps out the future, who knows the ecosystem — how do we sort of know that these are the ones that will figure out how to tip the market in their favor? What are some of the indicators — it’s not an accident. Like, they’re pulling levers strategically.

      Prof. Arthur: Yeah. Let me give you an analogy. Shows how hard this is to predict. I remember sitting — in 1991, I was invited to the Senate building to brief Al Gore, who was a senator then. It was an afternoon, and was quite hot. And they’re all sitting there, everybody was a little bit sleepy. And then Gore says, “Can you give me an example I can latch on to?” And I said, “Yeah, presidential primaries.” <laughter> And they got it immediately. The phenomenon I’m talking about, you know — if something gets ahead, it tends to get further ahead. It’s true in presidential primaries, that if some candidate pulls ahead, they get more financial backing, they can be more visible. The more visible they are, the more likely it looks that they might win the presidency.

      So they get further ahead and more backers. You have to be quite a way into the game before it’s pretty clear. That’s the best I can do on that. Meaning, sometimes if there’s a very early tilt, like, within a few months, it’s pretty clear what’s gonna take over. But it can be very much like presidential primaries. It’s all the same mechanism. And predicting exactly who that’s going to be might look easy afterwards, but on the spot, it’s very difficult to do.

      Marc: Well, this goes, actually, to the nature, I think, of how history is written, right? Which is, the way history gets written is, the victor is imputed all kinds of positive qualities. Like, genius, visionary, marvelous executer, right? And everybody knew, right? Everybody predicted. And then, of course, the people who don’t win, it’s like, “Oh, idiot, you know, losers, what were they thinking?”

      Prof. Arthur: Yeah. Exactly.

      Marc: We experience this in venture capital. It’s like, we basically get two kinds of press coverage. One is what a bunch of geniuses we were for backing the successful company, and what a bunch of morons we were for backing the failing company. And I keep pointing out we’re the same people. We don’t whip between genius and moron. We’re somewhere in the middle. But to your point, it’s the nature of the technology casino. The other thing I’ve observed on this point is — I don’t know if it’s cynical sense, or maybe a realistic sense — in a sense, the question of, like, what is the spark that causes one to jump ahead? In a sense, it kind of doesn’t matter.

      Sonal: Oh, that’s actually, kind of, very sacrilegious.

      Marc: Or even say, a less cynical way to put it might be — there might be 20 different ways somebody gets that initial jump. It might be they start two months earlier, it might be they raise a little more money, it might be they get a key distribution partnership. It kind of doesn’t matter exactly what it is as long as there is — as long as something actually happens. And so, there is a lot of idiosyncratic kind of history to these things.

      Prof. Arthur: Yeah. And my shorthand term for all that is luck. Of course, there’s no such thing. It’s just all small events. Who sat beside whom in a airplane, and chatted up somebody or whatever.

      Marc: Or whose mother happened to be on the board of United Way — the CEO of IBM, as one example.

      Prof. Arthur: Yes. Yeah, yeah. Famous story. Yeah. Yeah.

      Sonal: Oh, right. This is that infamous Bill Gates biography story — where, because his mom was on the board of United Way, she met the CEO of IBM, and then that meeting led to Microsoft and IBM striking a software deal that helped Microsoft in the early days.

      Marc: Right. Exactly right. Right. The other interesting kind of situation that we run into a lot on this, when we try to figure this out is — it’s fairly often you’ll have a scenario where you’ll have 2 — you might have 20 in the field, but you’ll have 2 companies that you kind of think have the highest probability of winning. And one of them is a little bit further ahead, but has a somewhat less-skilled or experienced founder. And then you’ll have another company that maybe started a little bit later, that will be further behind for the moment, but has a much more experienced and qualified founder CEO. And if you’re going entirely based on current trends, you go with the less-experienced, less-knowledgeable founder. On the other hand, you often have somebody very sharp who’s like, “Oh, yeah, I know exactly what I’m doing. He doesn’t know what he’s doing. I can take him out.” And like, that’s a real — that’s a conundrum that we face every day. <crosstalk> And it really elevates this kind of question of, like, how important actually is skill?

      Prof. Arthur: I mean, you’ve pretty much answered your own question, I think. Skill is extremely important, but it’s not tech skill. It’s not even skill in raising money. Those are kind of necessary, but not sufficient. What sort of skill is really, really important is strategic skill. It’s feel for how to build here, how to build up there. Basically, I often thought of this as surfing. You either get a wave or you don’t. If you get the wave, the whole momentum of that wave pulls you forward, and then you’ve got to maneuver and stay in the green water.

      How new technology is created

      Sonal: Oh, yeah. That was an analogy that Pete Pirolli used to use a lot at PARC for innovation, because he’s such an avid surfer. He would compare the two. And I remember reading an article years ago by JSB as well, that compared executives to surfers. 

      But let’s actually now shift gears and talk about, like, you know, once you understand these concepts that we’ve been talking about so far. Once you have these building blocks, like, network effects and positive versus diminishing returns — that you can essentially manipulate to pull levers and get the outcome you want, maybe luck, maybe not. The bigger question is, are there macro forces at play here too? I don’t mean macroeconomic. I mean more around the nature of technology and how it evolves — which coincidentally is the name of the book you published in 2009. I have a copy from you on my shelf. Anyway, it surprised me that you once argued that tech evolution is not like evolution in the obvious sense. So, tell us about that.

      Prof. Arthur: Well, yeah. Quite a while ago, about 15, 20 years ago, I got really interested in where technology comes from. And the idea around that we have is that there’s some genius in an attic or something.

      Sonal: Usually a garage.

      Prof. Arthur: Yeah, a garage. That’s right. Cooking up technology and coming up with inventions. What started to become clear to me as having looked in detail at some inventions — that technologies, in a way, come out of other technologies. If you take any individual technology, say, like, a computer in 1940s, it was made possible by having vacuum tubes. By having relay systems, by having very primitive memory systems, maybe mercury delay tubes. All of those things existed already. And so, it seemed to me that technologies evolved by people not so much discovering something new, or inventing, but by putting together different Lego blocks, so to speak, in a new way. Once something was put together — like, say, a radio circuit for transmitting radio waves — it could be thrown back in the Lego set. And occasionally then, some of the new combinations would get a name and be tossed back in. Things like gene sequencing were put together from existing molecular biology technologies, and then that becomes a component in yet other technologies…

      Sonal: Right. I mean, CRISPR is a great example.

      Prof. Arthur: CRISPR. Exactly.

      Sonal: Now you have CRISPR, which itself is a gene-editing tool, which then creates so many other things.

      Prof. Arthur: Yep. And that tool will be a component in future technologies. And I began to realize this wasn’t Darwinian. It wasn’t Darwin’s mechanism. It’s evolution, but it wasn’t that you vary radial circuits, or you vary air piston engines…

      Sonal: Right. It’s not like a natural selection effect.

      Prof. Arthur: Yeah. You can’t vary radial circuits and then suddenly get a computer out of that, or radar. You can’t vary air piston engines in 1930 and get a jet engine out of that. These things come along as completely new combinations, using new principles, and that keeps adding to your Lego set. And that starts to explain why there’s a controversy or a question, say, in the 1920s. Anthropologists were asking, why don’t you have trams and steam engines in the Trobriand Islands? And they began to say, “Well, it’s not because the islanders are stupid. It’s because they don’t have these building blocks to build it out of.” And that, in turn, has many implications. One of them is, if you get a region like Silicon Valley, with an enormous number of these building blocks — and more important, it has people who understand the craft to put all this together — not just the science, but what parameters — then it can very quickly keep coming up with new combinations.

      Sonal: What’s the implication for adoption though, for industry?

      Prof. Arthur: The implication is that if you have a new collection of technologies — let me just mention AI, artificial intelligence — those are all building blocks. Industry doesn’t adopt AI. AI is a slew of technologies. It’s a new Lego set. Industry is using its own technologies. And what really happens is that industries — the medical industry, the healthcare industry, the aircraft industry, the financial industry — they encounter this new Lego set of AI, and they pick and choose components to create their own new things.

      Sonal: <inaudible> and recombine, right.

      Marc: One of the interesting, sort of, aspects of that, I find, is as a consequence of what you’re describing. There, it seems to me, is a long pre-history of almost any “new technology,” right? A couple favorite examples I have of that — the French had optical telegraphy working, I think, 40 years before other people figured out electromechanical telegraphy. So, literally, tubes of glass underground in Paris with light pulses going through, and this is like the 1820s or 1830s.

      Prof. Arthur: Really?

      Sonal: I had no idea.

      Marc: Super early. Another great example…

      Prof. Arthur: With telescopes or something?

      Marc: Yeah. Some sort of — I mean, they were, like, relay stations, but it was little flashes of light, like lanterns, through glass tubes. And so it was, sort of, fiber optics — 160, 180 years prematurely. My other favorite example is — MIT published a great book called “Tube” years ago. It was about the pre-history of television. And we think of television as being, like, 1930s, 1940s, Philo Farnsworth, all these guys. Well, it turns out the idea for television emerged immediately upon the idea for radio. And there was a Scottish inventor named John Logie Baird. And in the — I think 1910s — he invented mechanical television, because he couldn’t do the electric. He did mechanical.

      And so he literally had spinning wooden blocks. So he had pixels. It was almost like a computer display, but made out of, literally, wooden blocks. And the pixels would basically spin — the wooden blocks would spin to form pictures. And one of the funniest scenes in the book is, he takes it to the board of governors of the BBC, in 1912 or something, and they’re like, “You are completely out of your mind.” And he’s just like, “Oh, just let me prove it. Let me prove it. Let me get some sets out there. I’ll prove that people want to do this.” And they finally gave him a programming block. They gave him access to the radio frequency — Thursday night, midnight for 15 minutes — and he was broadcasting for months…

      Sonal: That’s amazing.

      Marc: …you know, mechanical TV that nobody ever saw, right? And then 30 years later, right, people picked it up and actually made the version that works. So then I go through all this just to kind of say — so then what we can project forward is that all of the breakthrough technologies of the next 30, 40, 50 years — they already, in a sense, exist in some form. Is that…

      Prof. Arthur: Yes, that’s right. Pretty much. To get a new technology, you need two things. You need to sort of have a principle — meaning a way of doing things. Early television worked on this idea that you could pick up pixels, or little snippets of light or darkness in the image you’re looking at, and then transmit those by radio — very high frequency — decode it at the other end, and reproduce on some screen or another. So, yeah, you need a principle and you need the components. There’s a famous example Stanford was involved in. In the very early 1900s, the U.S. Navy was very interested in telegraphy, or telegraph.

      What they had at the time was spark radio. So, you could sort of, you know, <sound of electrical current> send these Morse code things across the whole spectrum. Anybody could pick it up. So they were looking for a perfect sine wave, as continuous-time radio — a continuous wave, not just a spark wave — at a single frequency. There was a company formed, Pacific Telegraph, sometime around 1906, 1907. They managed to get the guy who invented the triode vacuum tube…

      Marc: Oh, de Forest?

      Prof. Arthur: Yeah, Lee de Forest. Came out from Yale, kind of on the run from predators. De Forest and Federal Telegraph spent several years trying to get a perfect sine wave so that they could transmit radio waves on a single frequency offshore to naval vessels. They couldn’t really do it. And in 1912, AT&T put out a call for inventions. Their idea was to be able to telephone from New York to Chicago, but you needed to have some sort of repeating circuitry — needed to clean up the wave every 20 miles or so, and then retransmit it. It turned out that within about six months, three inventors — de Forest among them — came up with a triode vacuum tube early amplifier. That amplifier was fed back — that becomes an oscillator, kind of like a microphone shrieking. The oscillator gives you a perfect sine wave. You could modulate that and send that out as a radio message to ships offshore, or to anything. 

      And if I recall right — this is quite early in the game. These radio guys with headphones — they were always called sparks, the radio officers in ships — they were listening to Morse code one time, not very far from here, and suddenly, somebody transmitted music. They all, kind of, jumped — what the hell, you know? <laughter> They’re listening to dot-dot, you know? <Beethoven.> And suddenly there’s music coming out of their headphones, and it blew their minds that this was possible. It’s a bit of a long story, but the point is that individual inventions, like the triode vacuum tube, when put together in clever ways with other components, give you an oscillator, which is the basis of radio transmission. They give you radio receivers, etc., and that builds up the broadcasting industry, which, in turn — parts of that are used to give you television. And then in relay form — on or off switches — these things start to give you logic circuits, and in turn, that gives you early computers, etc.

      Sonal: Which in turn… the semiconductor industry to — right.

      Prof. Arthur: So, technologies don’t come out of nowhere. They come out of a very deep understanding of what’s in the Lego box and how to put those things together.

      Marc: Well, so the pessimistic view on that would be — boy, that means by implication, there really aren’t the kind of eureka moments that people think about. And the pessimistic view on that is — then, therefore, there’s really not gonna be anybody sitting around in the next 20 years who’s gonna say, “I wanna build warp drive,” and therefore faster than light travel. And they’re just gonna come up with it. Or immortality, or whatever, you know, these — and so, in a sense, it’s an argument against, kind of, dramatic innovation. Let’s just say determined innovation. On the other hand, it’s an optimistic argument, because it says the number of combinations of the Lego blocks are combinatorially effectively infinite over time.

      Prof. Arthur: Well, I did argue that there are breakthroughs, you know, there are eureka moments. They tend to work that — I’m sitting here wondering how I could get some effect. How could I transmit images by radio wave? And I could be sitting there thinking for months, “Well, I could use this combination, that combination, and another combination,” and then suddenly I realize, if I can get this in place, and that in place, and the other thing in place, that’s gonna work. And the interesting thing is — and I’ve read individual accounts by the dozen from inventors, even lab books. You see this again and again, “Can’t do it. Can’t do it. Can’t do it,” and then, “Oh. Oh. Oh.” One of my favorite stories is that the steam engine already existed way before James Watt. And James Watt, in the 1760s — I think it was in Glasgow — was brought in to see if he could improve it.

      So Watt thinks it over and he thinks, “Oh, well, you know, you’re heating the steam, you’re expanding it in a cylinder, then you’re suddenly cooling it again, and all of this is pretty slow. What if I allowed the steam to expand the cylinder, and then that steam is ejected into a second cylinder that’s kept at [a] very low temperature?” Suddenly, the steam collapses, there’s a vacuum, etc. So he invented an independent cold cylinder. He thought of it passing the village green on a Sunday, the Sabbath day. He was properly Scottish. It nearly killed him. He says, “And there I was…” And he knows it’s gonna work, but he can’t get into his workshop until Monday. You can just read this stuff and see that it’s half killing him, that he can’t prove the concept until the next day. He’s a machinist, and he got it to work fairly readily.

      Sonal: So he’s using the building blocks to basically — people are using existing building blocks to do this sort of combinatorial innovation, combinatorial evolution…

      Prof. Arthur: That’s right. Yeah, yeah. The point I’m making is that new technologies don’t build up as just pure inventions. There’s plenty of breakthrough insights, but they build out of what’s already there, the components. And quite often, then, new things come along, some key breakthrough technologies. Deep learning is one. CRISPR is another.

      Sonal: Right. These aren’t just isolated components. They themselves are tools, and literally recombine or create other technologies. And, by the way — in that sense, I think it is very much like evolution. I mean, we had Yuval Harari on the podcast too. And basically, in his book “Sapiens,” he argues that tech helps mankind leapfrog natural evolution. And, only in that context — we were talking about it across a much larger timescale. But in this context, I do think of it as a primordial soup for the next phase. 

      On that note, you mentioned deep learning, which — we think of it as, basically, machine learning, distributed computing, artificial intelligence. I mean, just for this purpose, we can broadly clump that into one category. And I remember a big piece you did for “McKinsey Quarterly,” right before I left PARC. It was around 2011, and it was on the second economy. Basically an autonomy economy. And actually, you should summarize this, because then I’d like to talk to you about how you might update that today, given all the advances in AI since.

      Prof. Arthur: Sure. Yeah. What I was pointing out was that there’s a familiar physical economy, the one we all know about. It has to do with retail stores, and factories, and banks — all the stuff that we see in the physical world. I was checking into a flight in the San Jose airport, sometime around 2011, and when I put my frequent flyer card in, suddenly, it was triggering a lot of processes. Certainly, the flight was being alerted that I was now there, maybe TSA was being alerted. So I began to realize that somehow there’s a huge second economy out there of machines talking to machines. I was thinking of it as a very large underground, unseen, invisible economy — could be in the cloud — of servers talking to servers, of software and algorithms talking to servers, talking to other servers — all being transmitted and in conversation. Always on, and occasionally then, putting out shoots up into the physical world.

      And it reminded me as a metaphor of aspen trees. Aspen trees, apparently, are one huge organism — that is, they’re all connected underground with the same root system. And what you see on the surface is the trees themselves, but there is a very, very large underground root system that’s all connected. These roots are all talking to each other, and this would be like the second economy. I now think I should have chosen the term “virtual economy,” or better still, the “autonomous economy,” because all of this is happening without our knowing. It’s autonomous. It’s things talking to things. So I don’t emphasize an internet of things. It’s more like an internet of conversations. Things triggering things, things switching off things, and querying.

      Sonal: I mean, just to give it a quick picture. If you have that image of you putting the card in the kiosk at the airport, and you have all these machines talking to each other, if you were to light up all those machines at once, they’d be all around the world. There’d be servers on Amazon’s cloud, there’d be something local. The local printer. There’d be something else, like, a processing payment thing, maybe in Palo Alto. There could be all these different pieces kind of coming together to drive that one transaction.

      Prof. Arthur: Yes. And not just a few dozen computers or servers lighting up, because those servers would be lighting up other servers. <Right.> And so, in the end, there could be hundreds of thousands of servers that were lighting up very briefly, maybe only for a few fractions of a second, and then shutting down again, and then passing messages. So I was interested in this autonomous economy. There was general conversation about automation and robots, and 3D printing. And I thought, no, they’re missing the point. I tend to think that the digital revolution — I believe there is such a thing, and I believe it keeps morphing or changing. About every 20 years, the digital revolution gets a new theme. <Right.> And the latest revolution comes almost by accident — that in the 2010s or so, we started to get huge numbers of sensors. Sensing chemicals, sensing visual pixels, sensing images, sensing temperatures — by the hundreds and dozens and hundreds of thousands. And all these sensors out there — and they were maybe feeding back from smartphones or from your car, and huge amounts of data.

      About the same time, and this was no coincidence, along comes a new generation of neural networks powered by deep learning. But more than anything, powered by all the data that the sensors are bringing us. And these algorithms started to be able to do one thing very well, and that was pattern recognition. Could recognize your voice much better than before, because of all the data, all the training. It could recognize faces. So, suddenly, we got the ability of algorithms to do things that we thought only humans could do.

      As recently as 20 years ago, or 10 years ago, we would have said, “Oh, yeah, computers are great, but they’ll never be good at what humans are good at.” What are humans good at? We’re good at recognizing things, we’re good at fast association. Computers, they can do deduction or logic. We’re not much good at logic, so it seemed that the whole world was nicely divided.

      Sonal: But now…

      Prof. Arthur: But now, computers have learned to do associative thinking. These patterns mean such and such. And so, suddenly, we’re in an area that we thought only human beings were gonna be good at, and we’re seeing industry after industry change as a result. It’s not just automation, it’s much more than that. It’s redoing or restructuring whole areas of the economy. So, I was looking for an analogy in history that even vaguely resembles what’s happening. The printing revolution, starting around the 1450s — suddenly information went from being very closely guarded by monasteries and abbeys and libraries — these big vellum books chained to desks. And with printing, it became publicly available. So, printing made information externally available, and that changed everything. It very much changes the way people are thinking. Copernicus, for example, had at his disposal data that he could not have got hold of if it just existed in monasteries. It made a huge difference. It brought in modern science, it helped the Renaissance, and this brought us our modern world.

      Sonal: I mean, I would agree — but is that the big transformation now? That we have the modern tech equivalent of the printing press?

      Prof. Arthur: What’s gone external now is not information. What’s gone external is intelligence. I may be driving in a convoy of 50 driverless cars, and the whole idea of the car adjusting — the car is talking to roadside sensors and servers, it’s talking to other cars, it’s talking to the highway patrol servers, and so on. And it’s basically farming out its intelligence into this other economy, and then getting back intelligent actions in return. So it’s a bit like phone-a-friend, only the friend is incredibly smart, and the friend consists of, again, these hundreds of thousands of servers talking to each other and then adjusting what you do. So suddenly, intelligence doesn’t just exist in human beings. Suddenly, intelligence exists in the cloud, or in this autonomous economy, and we can farm out not just getting information, but getting smart moves back. And this is making all the difference.

      Sonal: So it’s not about the form intelligence takes, it’s that intelligence is no longer housed internally in the brains of human workers. Because it’s moved outward into the virtual economy.

      Prof. Arthur: Yes, that’s right.

      Impact of computer intelligence

      Sonal: So, when intelligence is not just information, but, sort of, decision-making, or being able to externalize a lot of this — I mean, one of the things you mentioned earlier is about these building blocks of technology. What happens when all of these things are available to everybody equally? Like, is there not, like, a sort of a Red Queen effect, where everyone’s accessing the same building blocks and tools? So how do companies, how do industries find competitive advantage in that kind of a world?

      Prof. Arthur: I think the answer to that question is timing. If I’m a retail bank, whatever that might be, I might be quite a large bank. And I’m saying all these externally intelligent technologies and algorithms are suddenly available. How can I make use of that, and how can I bring those into my operations and combine them with what I’m doing? I’m making mortgage loans, I’m acting as escrow or something, you know — all these various different types of financial operations. I can make a lot of them automatic and autonomous, and get an advantage. The trouble is that that can be rapidly commoditized.

      Sonal: So what does that mean for jobs? In this podcast, we talk a lot about how whenever industries are changed in this way — you know, through tech and other shifts — that other new jobs — classic examples include more designers in the age of Adobe design, that new jobs that never existed before, like social media managers — that can only exist today. What’s your take here?

      Prof. Arthur: So, what I’m seeing is — about 90 years ago or so, John Maynard Keynes pointed out that he thought by 100 years’ time, 2030, we’d be in an economy where the production problem was largely solved. There’d be enough, in principle, to go around for everyone. There might be plenty, in principle — goods and services around — but getting access to them meant you needed wages, which you needed a job for, and that was not possible. I think that what Keynes said in that regard is becoming true. In other words, the trough is full, but how do the piggies get their share of the trough? So we’re now in a new distributive era. 

      What’s counting is not how much is produced, but who gets what. The whole question of growth and getting more economic product out there — physical product and services — that’s a job for entrepreneurs, and it’s a job for engineers. Who gets what is much more a political issue, and that’s not quite a job just for politicians, but it’s a job for society to solve. And we haven’t solved it in Europe or anywhere else, so it’s a new era.

      Marc: The problem with that theory is the same problem as that theory in Keynes’s era, right? Which is, sort of, Milton Friedman’s observation in the 1950s, 1960s, when that issue came up again. Which is that human wants and needs are infinite, right? One of the things we are best at as a species is coming up with new things that we want. <crosstalk> And then the things that we want in one generation become the things that we need in the next generation, right? Air conditioning goes from being a luxury to being something that we expect…

      Sonal: Cell phones…

      Marc: …and we get outraged when we don’t have [it], and cell phones and everything else. And, you know, he speculated as a thought experiment — he said, “Look, you know, we have no way of envisioning the wants and needs of what people will have in the future. We just know they’ll be there.” And he said, “Look, maybe it’ll be that, like, you know, right now, psychiatry is a luxury good. And maybe in the future, it’ll be a basic human right to have access to a psychiatrist, and then we’ll employ half the population being psychiatrists to the other half.” It just is one example, right, of…

      Prof. Arthur: I’m looking forward to this new economy.

      Sonal: I like that one. The best, actually, among the examples, so far.

      Marc: Exactly. And in economic terms, of course, the problem with Keynes’s analysis was, it overlooks the role of productivity growth, right? Which is the scenario that you were describing — is the scenario of, like — rapidly increase your productivity growth. And in a world of rapidly increasing productivity growth, you have gigantic gains in economic welfare. You have gigantic growth in underlying industries, right? You have gigantic amounts of entrepreneurial activity that come out of that. And that, then, generates a fountain of new jobs to satisfy all those new wants and needs. And then, finally, I can’t resist pointing out that you’re making this argument on a day when the unemployment rate in the U.S. dropped below 4%.

      There’s certainly no trace — and remember, once there was a day in the American economy — you actually had very low productivity growth, not very high productivity growth. Which counters against the argument that there’s some level of unprecedented technological disruption that’s happening, because you certainly can’t see it in the numbers. And then you have unprecedented levels of job growth and employment. So the facts seem to be on the other side of this argument.

      Prof. Arthur: Well, let me both agree and disagree here. I certainly agree that there will be whole new categories of jobs. I very much like the idea that half of us will be therapists.

      Sonal: I love that one too.

      Prof. Arthur: And the other half — and we can swap couches.

      Marc: Oh, yeah, no, no, the therapists will need therapists.

      Prof. Arthur: I think there’ll be plenty of new jobs invented. At the same time, though, not just through automation and not just through algorithms, but over the last 20 or 30 years, we’ve had a huge amount of globalization. Jobs have been off-shored, and that’s not just due to the rise of China. It’s due to the rise of telecommunications. Of — I can keep track of all the suppliers in China, all the factories in China, the inventories, and so on, in real time. Couldn’t have done that much in the 1980s because the technology wasn’t there. And that hollowed out an enormous amount of traditional workers in the middle of America, and certainly in Britain and in other countries. So, where I would come out on this question — I like your observation. I agree, yes, we will get new jobs, but quite often there’s a big lag in between the original happening of hollowed-out industries, and then something taking its place.

      An analogy that I like is that in Britain in the 1850s. The economy was going gangbusters. New textile companies, the railways were just starting to kick in. There was all kinds of possibilities — steelworks, everything got suddenly very serious. And at the same time — so, there were people getting very rich, but at the same time, there was child labor, there were… 

      Sonal: The Dickensian world…

      Prof. Arthur: The whole Dickensian world of people almost being worked to death. Both are true. The economy is going gangbusters. Some people are not doing well out of this. It took about 30 to 60 years before the whole thing equalized and workers had safe conditions, they had much better conditions — and eventually, they were able to partake in a decent way in all this wealth creation. So what I would say is that the digital economy, through globalization — and now through algorithms — is pressing us into a scramble to invent new categories of jobs. I’m optimistic. I think eventually, we’ll get on top of this. And I’m hoping we do it in a good way, where we have creative pursuits, not just rote jobs, like we might’ve had 100 years ago. I think things are going quite well.

      Marc: Good.

      Globalization and international affairs

      Sonal: So, it is a global world now, and it depends on what your frame of reference is. For me, my frame of reference is — I have relatives in India, who are now increasing in their middle class. If your frame of reference is global, you see this as a very different kind of shift. It really depends on where you, sort of, put the square, the rectangle of the frame, and where you zoom in. Because there is Africa, you know, another great example, Cambodia. You have all these countries — there’s something interesting happening there. So, speaking of that, I’d love to hear — because you spent a lot of time in Singapore. I’d love to hear your thoughts on, sort of, the evolution of that, because we’ve often made the argument that this kind of form — top-down, government-planned innovation cluster — never works out, and Singapore is a rare exception. How would you distill it, having been on the ground there?

      Prof. Arthur: I’m a watcher of countries that look as if they’re in trouble and then make their way out of trouble. Finland’s a good example, because The Cold War shuts down, Finland was a broker — a bit like Hong Kong, in between the West and the East. Then around 1990, suddenly the bridge is there, but the river ceases to exist. And so then they invented their way out of that with Nokia and other companies. Their back was to the wall. And I could say the same thing in Singapore. When the country was set up, about 51 or 52 years ago, they felt very much as if they had been set adrift — so, like a little rowing boat that was being towed behind Malaysia, and then somebody cut the rope. So, I think, again, it was a matter of desperation, very good planning. People like Lee Kuan Yew, who led the government.

      And what they did was, they decided that they would go into what was then tech manufacturing. They had inherited shipyards from the British, etc., so they were able to station themselves as a very early manufacturer, a bit like Hong Kong or Taiwan. Produce cheap goods, and take great advantage that the oil tankers had to stop at — and become a commercial and brokerage hub for shipping. Since that, they’ve moved into finance. What I’m finding — and let me broaden into Asian countries, including China — we tend to think of — as recently as 10 years ago, we would have thought of China as being not fully developed. Not at all like Japan, which is developed. Singapore is quite developed.

      What we’re now seeing in Asia is that a lot of countries in Asia, including China, their digital revolution is not much more than two to three years behind what’s happening in California or in the West. They’re extremely well-advanced, they’re paying a huge amount of attention to technical education, and it’s not just that they’re following in China. They’re not just following, say, genomics or AI. They’re inventing their own. Singapore, by dint of strong will and going techie, has managed to do that already. What I do notice in Singapore is that they tend to — not so much initiate perfectly new technologies, but they’re very quick to take them up. China, though, is able to initiate things…

      Sonal: Initiate them as well, right.

      Prof. Arthur: …especially in things like genomics.

      Sonal: Do you think the initiation thing matters? Because part of your thesis around there being these building blocks that are widely available, which leads to this combinatorial innovation, combinatorial evolution, as you describe it. I wonder if that even matters so much anymore, because if these building blocks — open source, APIs — all are available. Like, application programming interfaces that people can combine into entirely new companies. It seems like you can actually draw on the best of the best expertise.

      Prof. Arthur: I think so. That’s been a long debate, actually, in economics. Why put all the effort into initiating something when you can just position yourself to learn the technology quickly? The other case is, it’s good to be first. I think it’s debatable. What I would say though in China is that when it comes to a country digitizing everything, China isn’t going to be far behind.

      Sonal: It’s especially true of AI, actually.

      Prof. Arthur: Yes. Especially in artificial intelligence and in genomics, and probably in several other industries.

      Sonal: Well, genomics is particularly interesting, because they were the first to do human-scale studies of CRISPR. Because we, regulatorily — rightly so — may not be able to, or maybe not. I don’t know. I don’t have an opinion on that.

      Prof. Arthur: What I see is this sort of technology expanding rapidly into the rest of the world — and the other country, of course, to mention is India. For several decades, technological education in India has been excellent. IIT, places like that, Bangalore — and India is not very far behind. China’s in a better position because China is top-down hierarchical. They can quickly reorganize and change their economy.

      Sonal: We go back and forth around this all the time, but every past industrial planning, top-down, centralized model of coordination has eventually eaten its own, and fallen on its own, like — hoisted on its own petard, to use that expression. Which is kind of the thing that inevitably seems to happen. That’s what happened in Japan, and it’ll inevitably happen with China.

      Prof. Arthur: Well, it may inevitably happen in the United States, too.

      Sonal: That’s true. That’s a very good point.

      Prof. Arthur: I do think that, occasionally, economies get a bit tired, people get complacent, etc. I was in India — I’ve been there several times. But a long time ago, like 1975, when there were old English cars — Morris Minors driving around, taxis that you wouldn’t have seen since the 1950s in London. And the Indian economy has gone light years beyond that.

      Sonal: Well, I would say that one of the other shifts there, which is important to note here for this part of the conversation, is that India, China, Singapore — they’ve moved away. Well, India went through an outsourcing phase, as you know, you described this, to being originators of their own innovation. They’re not just a copycat narrative. And we’ve written about this when it comes to China as well. I mean, just yesterday, Walmart announced it’s buying Flipkart — which that’s, kind of, an inversion of the typical model that would have happened before. So, anyway, I think that’s an important shift that this is playing out against.

      Prof. Arthur: Yeah. The rest of the world is very rapidly catching up. I still think that the U.S. economy is going to do extremely well.

      Sonal: That’s great. It’s optimistic.

      Prof. Arthur: Well, it’s not just optimism. I think it’s pretty well inevitable. Let me restate this. I think what’s gonna happen in the next decade or two — the story in the U.S. economy is simply going to be that huge industries are going to reorganize themselves along the lines of autonomous intelligence.

      Sonal: When you described that the economy has these, sort of, 20-year beams that you’ve seen, and you’ve described them as morphings in your writings. Like, sort of fundamental sea changes. And you described integrated circuits already, and fast computation as the first — we talked about the connection of digital processes, and now you mentioned these sensors. The cheap and ubiquitous sensors. My question for you, as someone who’s long studied this, is — how do you know when you’re seeing the beginning of one of these revolutions? That it’s a morphing in the making. Is this, sort of, a hindsight view? Because you are, sort of, seeing it early with everything else. What are the signs that tell you this is a morphing — this is a big theme that’s emerging? That gives you the confidence to say that about, say, deep learning, or CRISPR, even?

      Prof. Arthur: I think that a change is usually quite well underway before people pick it up. You wake up one day and you say, “Oh, my god. The game has changed.” In the case of sensors, I remember in 2010 or so, sitting down with the CTO of Intel, and I asked him, “Can you tell me when the average sensor is, for example, at a parking meter — that might sense a car being at the meter — the average sensor is gonna drop below about 10 cents per unit?” And he said, “Yeah, that’ll be around 2013, 2015.” He knew pretty well exactly, and so I thought, that’s gonna be a game-changer, because we will now know what’s happening everywhere. What I didn’t see at the time was that the ubiquity of sensors would bring in big data. Some of us saw that in advance, but the big data — [we] didn’t see [it] would bring in all these smart algorithms. <Right.> And so, it’s the combination. Is there a way to see these new things coming along? Yeah. If you’re waiting for them.

      Sonal: This reminds me of a story that Alvy Ray Smith tells. He’s a PARC alum as well. He co-founded Pixar back in the day. And he did a piece for me at “WIRED” about how they knew very early on — they had John Lasseter, they had this creative vision — they knew very early on the kinds of things that they wanted to do. And they later mapped out, like, a trajectory of their movies based on Moore’s law, but it was, like, a tool for them. So they saw it, but yeah, they had to wait.

      Prof. Arthur: But usually, it’s hard to see. The best I can hope for, at least in my own case, is that within two or three years, you just go, “Oh, the game has changed.” And when the game changes, you realize you’re in a slightly different era. And when you’re in that era, you realize that it’s not gonna last. That in 10 years’ time, 20 years’ time, or 30 years’ time, there’ll be a different version. I wanna make this comment very quickly. I’ve been physically in Silicon Valley — if you count Berkeley, I’ve been in the…

      Sonal: I think we should count Berkeley.

      Prof. Arthur: Yeah. Okay. I’ve been in the Bay Area now for very close to 50 years. I was a grad student in Berkeley. And then, in Stanford, I’ve been here since 1982. And in all that time, when the game gets a little tired at times, people say, “Oh, the Valley is over,” but it doesn’t. It discovers new technologies and then reinvents itself. That’s the way capitalism works here in Silicon Valley, but in other countries, where it’s more planned, it may have stopped, and places like that can come to a halt as a result.

      Sonal: That’s the perfect note to end on. I’m gonna quote a piece from one of your middle-early papers. You talk about whether there’s any hope in complexity, essentially, and you say, “It shows us an economy perpetually inventing itself, perpetually creating possibilities for exploitation, perpetually open to a response. An economy that is not dead, static, timeless, and perfect, but one that is alive, ever-changing, organic, and full of messy vitality.”

      Prof. Arthur: It’s not a coincidence that I wrote that, because that’s the way Silicon Valley operates. Inventing and reinventing itself, and morphing and changing, in a way you can’t quite predict, and in a way that I think is delightfully messy, but ordered at the same time.

      Sonal: Fabulous. A messy ordered vitality. Brian Arthur, thank you for joining the “a16z Podcast.”

      Marc: Yeah. Thank you, Brian. That was really tremendous.

      Prof. Arthur: And thank you very much for having me. I’m delighted. Thank you.

      Sonal: Thank you.

      • W. Brian Arthur

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Feedback Loops — Company Culture, Change, and DevOps

      Nicole Forsgren, Jez Humble, and Sonal Chokshi

      From the old claim that “IT doesn’t matter” and question of whether tech truly drives organizational performance, we’ve been consumed with figuring out how to measure — and predict — the output and outcomes, the performance and productivity of software. It’s not useful to talk about what happens in one isolated team or successful company; we need to be able to make it happen at any company — of any size, industry vertical, or architecture/tech stack. But can we break the false dichotomy of performance vs. speed; is it possible to have it all?

      This episode of the a16z Podcast boldly goes where no man has gone before — trying to answer those elusive questions — by drawing on one of the largest, large-scale studies of software and organizational performance out there, as presented in the new book, Accelerate: The Science of Lean Software and DevOps — Building and Scaling High Performing Technology Organizations by Nicole Forsgren, Jez Humble, and Gene Kim. Forsgren (co-founder and CEO at DevOps Research and Assessment – DORA; PhD in Management Information Systems; formerly at IBM) and Humble (co-founder and CTO at DORA; formerly at 18F; and co-author of The DevOps Handbook, Lean Enterprise, and Continuous Delivery) share the latest findings about what drives performance in companies of all kinds.

      But what is DevOps, really? And beyond the definitions and history, where does DevOps fit into the broader history and landscape of other tech movements (such as lean manufacturing, agile development, lean startups, microservices)? Finally, what kinds of companies are truly receptive to change, beyond so-called organizational “maturity” scores? And for pete’s sake, can we figure out how to measure software productivity already?? All this and more in this episode!

      Show Notes

      • Where DevOps originated historically [1:58] and why it’s important for businesses today [5:45]
      • The practicality of analyzing DevOps [7:42] and the results of research to determine DevOps best practices [11:50]
      • Discussion around ideal DevOps structures and differences across company types [16:34]
      • What the data show about measuring and improving productivity [20:37]
      • How companies determine whether they’re ready to implement recommended changes [30:24] and the importance of culture and leadership [36:47]

      Transcript

      Hi, everyone, welcome to the “a16z Podcast.” I’m Sonal. So, one of our favorite themes to talk about in this podcast is how software changes organizations, and the nature of the firm. And today’s episode takes a different angle on the topic by drawing on the research of one of the largest large-scale studies of software and organizational performance out there. From the authors of the new book “Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations,” by Nicole Forsgren, Jez Humble, and Gene Kim.

      Joining us to have this conversation, we have Nicole, who did her Ph.D research trying to answer the elusive eternal questions around how to measure this, especially given past debates around “does IT matter?” She’s now the CEO of DORA, DevOps Research and Assessment, which also puts out, with Puppet, the annual State of DevOps Report.

      And then we have Jez Humble, who is CTO at DORA, and is also the co-author of the books, “The DevOps Handbook,” “Lean Enterprise,” and “Continuous Delivery.” They share the latest findings about high-performing companies of all kinds, even those who may not think they’re tech companies, and answer whether there’s an ideal organizational type in size, architecture, culture, or people that lends itself to success here. But we began this episode by briefly talking about the history of DevOps, and where it fits in the broader landscape of related software movements.

      Dr. Forsgren: So, I started as a software engineer at IBM. I did hardware and software performance. And then I took a bit of a detour into academia, because I wanted to understand how to really measure and look at performance that would be generalizable to several teams in predictable ways, and in predictive ways. And so, I was looking at and investigating how to develop and deliver software in ways that were impactful to individuals, teams, and organizations. And then I pivoted back into industry, because I realized this movement had gained so much momentum and so much traction. And industry was just desperate to really understand — what types of things are really driving performance outcomes and excellence?

      Sonal: And what do you mean by this movement?

      Historical background

      Dr. Forsgren: This movement that now we call DevOps — so the ability to leverage software, to deliver value to customers, to organizations, to stakeholders.

      Jez: And I think, from a historical point of view, the best way to think about DevOps, it’s a bunch of people who had to solve this problem of how do we build large distributed systems that were secure, and scalable — and be able to change them really rapidly and evolve them? And no one had that problem before, certainly at the scale of companies like Amazon, and Google. And that really is where the DevOps movement came from, trying to solve that problem. And you can make an analogy to what Agile was about, since the kind of software crisis of the 1960s, and people trying to build these defense systems at large scale — the invention of software engineering, as a field, Margaret Hamilton, her work at MIT on the Apollo program. What happened in the decades after that was, everything became kind of encased in concrete in these very complex processes: “This is how you develop software.” And Agile was kind of a reaction to that, saying we can develop software much more quickly with much smaller teams in a much more lightweight way.

      Sonal: So, we didn’t call it DevOps back then but it’s also more agile. Can you guys break down the taxonomy for a moment? Because when I think of DevOps, I think of it in the context of the containerization of code, and virtualization. I think of it in the context of microservices, being able to do modular teams around different things. There’s an organizational element, there’s a software element, there’s an infrastructure component. Like, paint the big picture for me of those building blocks and how they all kind of fit together.

      Jez: Well, I can give you a very personal story, which was my first job after college was in 2000, in London, working at a startup where I was one of two technical people in the startup. And I would deploy to production, by FTP and code from my laptop going into production. <yeah> And if I wanted to roll back, I’d say, “Hey, Johnny, can you FTP your copy of this file to production,” and that was our rollback process. And then I went to work in consultancy where we were on these huge teams and deploying to production, there was a whole team with a Gantt chart, which put together the plan to deploy to production. And I’m like, “This is crazy.” Unfortunately, I was working with a bunch of other people who also thought it was crazy.

      And we came up with these ideas around deployment automation, and scripting, and stuff like that. And suddenly, we saw the same ideas have popped up everywhere, basically. I mean, it’s realizing that if you’re working in a large, complex organization, Agile is gonna hit a brick wall. Because, unlike the things we were building in the ’60s, product development means that things are changing and evolving all the time. So it’s not good enough to get to production the first time, you’ve got to be able to keep getting there, on and on. And that really is where DevOps comes in. So it’s like, with Agile we’ve got a way to build novel products, but how do we keep deploying to production, and running the systems in production in a stable, reliable way, particularly in a distributed context?

      Dr. Forsgren: So, if I phrase it another way, sometimes there’s a joke that says, “Day one is short and day two is long.”

      Sonal: What does that mean?

      Dr. Forsgren: Right. So day one is when we, like, create all this…

      Sonal: By the way, it’s sad that you have to explain the joke to me. <crosstalk> But you do actually have to explain that joke to me. <laughter>

      Dr. Forsgren: No, which is great, though, because — so day one is when we create all of these systems, and day two is when we deploy to production. We have to deploy and maintain forever and ever and ever and ever.

      Sonal: So day two is an infinite day.

      Dr. Forsgren: Right. Exactly. And…

      Jez: For successful products.

      Dr. Forsgren: Hopefully. We hope that day two is really, really long. And we’re fond of saying Agile doesn’t scale. And sometimes I’ll say this and people shoot laser beams out of their eyes. But when we think about it, Agile was meant for development, just like Jez said, it speeds up development. But then you have to hand it over, and especially infrastructure, and IT operations — what happens when we get there? So DevOps was sort of born out of this movement, and it was originally called Agile System Administration. And so then DevOps sort of came out of development and operations. And it’s not just dev and ops, but if we think about it, that’s sort of the bookends of this entire process.

      Sonal: Well, it’s actually like day one and day two combined into one phrase.

      Dr. Forsgren: Day one and day two.

      Why DevOps is essential

      Sonal: The way I think about this is I remember the stories of, like, Microsoft and the early days, and the waterfall cascading model of development. Leslie Lamport once wrote a piece for me about why software should be developed like houses, because you need a blueprint. And I’m not a software developer, but it felt like a very kind of old way of looking at the world of code.

      Jez: I hate that metaphor.

      Sonal: Tell me why.

      Jez: If the thing you’re building has well-understood characteristics, it makes sense. So if you’re building a truss bridge, for example, there’s well-known understood models of building truss bridges. You plug the parameters into the model, and then you get a truss bridge and it stays up. Have you been to Sagrada Familia in Barcelona?

      Sonal: Oh, I love Gaudi. Yeah.

      Jez: Okay, so if you go into the crypt of the Sagrada Familia, you’ll see his workshop. And there’s a picture where, in fact, a model that he built of the Sagrada Familia, but upside down with the weight simulating the stresses. And so he would build all these prototypes and small prototypes because he was fundamentally designing a new way of building. <Oh, right.> All Gaudi’s designs were hyperbolic curves and parabolic curves, and no one had used that before.

      Sonal: Things that had never been pressure tested, literally, in that case.

      Jez: Exactly. He didn’t want them to fall down. So he built all these prototypes and did all this stuff.

      Sonal: He built his blueprint as he went by building and trying it out, which is a very rapid prototyping kind of model.

      Jez: Absolutely. So, in the situation where the thing you’re building has known characteristics, and it’s been done before, yeah, sure, we can take a very phased approach to it. And you know, for designing these kinds of protocols that have to work in a distributed context, and you can actually do formal proofs of them, again, that makes sense. But when we’re building products and services where, particularly, we don’t know what customers actually want, and what users actually want, it doesn’t make sense to do that. Because you’ll build something that no one wants, you can’t predict.

      Dr. Forsgren: And we’re particularly bad at that, by the way. Even companies like Microsoft, where they are very good at understanding what their customer base looks like. They have a very mature product line. Ronny Kohavi has done studies there and only about 1/3 of the well-designed features deliver value.

      The practicalities of DevOps analysis

      Sonal: That’s actually a really important point. The mere question of — does this work? — is something that people really clearly don’t pause to ask. But I do have a question for you guys, a push back, which is — is this a little bit of the cult, “Oh, my God, it’s like so developer-centric, let’s be agile, let’s do it fast, our way — you know, two pizzas, that’s the ideal size of a software team.” And, you know, I’m not trying to mock it. I’m just saying that, isn’t there an element of actual practical realities, like technical debt and accruing a mess, underneath all your code? And a system that, you may be there for two or three years and you can go off to the next startup, but okay, someone else has to clean up your mess. Tell me about how this fits into that big picture.

      Dr. Forsgren: This is what enables all of that.

      Sonal: Oh, right. Interesting. So it’s not actually just creating a problem because that’s how I’m kind of hearing it.

      Dr. Forsgren: No, absolutely. So you still need development, you still need tests, you still need QA, you still need operations, you still need to deal with technical debt, you still need to deal with re-architecting really difficult large monolithic code bases. What this enables you to do is to find the problems, address them quickly, move forward…

      Jez: I think that the problem that a lot of people have is that we’re so used to couching these things as trade-offs and as dichotomies, the idea that if you’re gonna move fast, you’re gonna break things. The one thing, which I always say is if, you know, if you take one thing away from DevOps is this, high-performing companies don’t make those trade-offs. They’re not going fast and breaking things, they’re going fast and making more stable, more high-quality systems. And this is one of the key results in the book, in our research, is this fact that high performers do better at everything, because the capabilities that enable high performance in one field, if done right, enable it in other fields.

      So, if you’re using version control for software, you should also be using version control for your production infrastructure. If there’s a problem in production, we can reproduce the state of the production environment in a disaster recovery scenario — again, in a predictable way — that’s repeatable. I think it’s important to point out that this is something that happened in manufacturing as well.

      Sonal: Give it to me. I love when people talk about software as drawn from hardware analogies. That’s my favorite type of metaphor.

      Jez: Okay. So I’ve got — so Toyota didn’t win by making shitty cars faster. They won by making higher-quality cars faster and having a shorter time to market.

      Sonal: The Lean manufacturing method, which by the way, also spawned Lean startup thinking and everything else connected to it.

      Dr. Forsgren: And DevOps pulls very strongly from Lean methodologies.

      Research methods and findings

      Sonal: So, you guys are probably the only people to have actually done a large-scale study of organizations adopting DevOps? What is your research and what did you find?

      Dr. Forsgren: Sure, so the research really is the largest investigation of DevOps practices around the world. We have over 23,000 data points, all industries…

      Sonal: Give me, like, a sampling — like, what are the range of industries?

      Dr. Forsgren: So I’ve got entertainment, I’ve got finance, I have healthcare and pharma, I have technology…

      Jez: Government.

      Dr. Forsgren: Government, education.

      Sonal: You guys have every vertical. And then when you take it around the world…

      Dr. Forsgren: So we’re primarily in North America, we’re in AMEA, we have India, we have a small sample in Africa…

      Sonal: Right. Just to quickly break down like the survey methodology, questions that people have. In the ethnographic world, the way we would approach it is that you can never trust what people say they do, you have to watch what they do. However, it is absolutely true, and especially in a more scalable sense, that there are really smart surveys that give you a shit ton of useful data.

      Dr. Forsgren: Yes. And part two of the book covers this in almost excruciating detail.

      Sonal: We like knowing methodologies, that’s nice to share that.

      Dr. Forsgren: Yes. Well, and it’s interesting, because Jez has talked about in his overview of Agile and how it changes so quickly, and we don’t have a really good definition. What that does is it makes it difficult to measure, right? And so, what we do is we’ve defined core constructs, core capabilities, so that we can then measure them. We go back to core ideas around things like automation, process, measurement, Lean principles. And then I’ll get that pilot set of data and I’ll run preliminary statistics to test for discriminant validity, convergent validity, composite reliability — make sure that it’s not testing what it’s not supposed to test. It is testing what it is supposed to test. Everyone is reading it consistently the same way that I think it’s testing, like, even run checks to make sure that I’m not inadvertently inserting bias or collecting bias, just because I’m getting all of my data from surveys.

      Sonal: Sounds pretty damn robust. So tell me, then, what were the big findings? That’s a huge question but give me the hit list.

      Dr. Forsgren: Well, okay, so let’s start with one thing that Jez already talked about: speed and stability go together.

      Sonal: This is where he was talking about there not being, necessarily, a false dichotomy. And that’s one of your findings, that you can actually accomplish both.

      Jez: Yeah. And it’s worth talking about how we measure those things as well. So we measure speed or tempo, as we call it in the book, or sometimes people call it throughput as well…

      Sonal: Which is a nice full-circle manufacturing idea, like the semiconductor, like, circuit throughput.

      Jez: Yeah, absolutely.

      Sonal: I love hardware analogies for software, I told you.

      Jez: A lot of it comes from Lean. So lead time, obviously one of the classic Lean manufacturing measures we use, how long does it take — we look at the lead time from checking into version control to release into production. <Right.> So that part of the value stream because that’s more focused on the DevOps end of things.

      Dr. Forsgren: And it’s highly predictable.

      Jez: The other one is release frequency. So how often do you do it? And then we’ve got two stability metrics, and one of them is time to restore. So in the event that you have some kind of outage, or some degradation in performance in production, how long does it take you to restore service. For a long time, we focused on not letting things break. And I think one of the changes, paradigm shifts, we’ve seen in the industry, particularly in DevOps is moving away from that. We accept that failure is inevitable because we’re building complex systems. So, not how do we prevent failure, but when failure inevitably occurs, how quickly can we detect and fix it?

      Dr. Forsgren: MTBF, right? Mean time between failures. If you only go down once a year, but you’re down for three days, and it’s on Black Friday — but if you’re down, very small, low blast, very, very small blast radius, and you can come back almost immediately and your customers almost don’t notice? That’s fine.

      Jez: The other piece around stability is, change fail rate. When you push a change into production, what percentage of the time do you have to fix it because something went wrong.

      Sonal: By the way, what does that tell you, if you have a change fail?

      Jez: So, in the Lean kind of discipline, this is called percent complete and accurate. And it’s a measure of a quality of your process. So in a high-quality process, when I do something for Nicole, Nicole can use it, rather than sending it back to me and say, “Hey, there’s a problem with this.” And in this particular case, what percentage of the time, when I deploy something to production, is there a problem because I didn’t test adequately, my testing environment wasn’t my production-like enough.

      Sonal: Those are the measures for finding this. But the big finding is that you can have speed and stability together through DevOps. Is that what I’m hearing?

      Dr. Forsgren: Yes, high performers get it all, low performers kind of suck at all of it, medium performers hang out in the middle. I’m not seeing trade-offs four years in a row — so anyone who’s thinking, “Oh, I can be more stable if I slow down,” I don’t see it.

      Sonal: It actually breaks a very commonly held kind of an urban legend around how people believe these things operate. So tell me, are there any other sort of findings like that because that’s very counterintuitive.

      Dr. Forsgren: Okay. So this one’s kind of fun. One is that this ability to develop and deliver software with speed and stability drives organizational performance. Now, here’s the thing…

      Sonal: I was about to say, that’s a very obvious thing to say.

      Dr. Forsgren: So, it seems obvious, right? Developing and delivering software with speed and stability drives things like profitability, productivity, market share. Okay, except, if we go back to Harvard Business Review, 2003, there’s a paper titled, “IT Doesn’t Matter.” We have decades of research — I wanna say at least 30 or 40 years of research — showing the technology does not drive organizational performance. It doesn’t drive ROI. And we are now starting to find other studies and other research that backs this up. Erik Brynjolfsson out of MIT, James Bessen out of Boston University, 2017.

      Sonal: Did you say, James Bessen?

      Dr. Forsgren: Yeah.

      Sonal: Oh, I used to edit him too.

      Dr. Forsgren: Yeah, it’s fantastic. Here’s why it’s different. Because before, right, in like the 80s, and the 90s, we did this thing where, like, you’d buy the tech and you’d plug it in, and you’d walk away.

      Sonal: It was an on-prem sales model where you, like, deliver and leave, as opposed to software as a service in other ways that things… 

      Dr. Forsgren: Yep.

      Jez: And people complain if you try to upgrade too often.

      Sonal: Oh, right. <laughter>

      Dr. Forsgren: The key is that everyone else can also buy the thing and plug it in and walk away. How is that driving value or differentiation for a company? If I just buy a laptop to help me do something faster, everyone else can buy a laptop to do the same thing faster. That doesn’t help me deliver value to my customers or to the market. It’s a point of parity not a point of distinction.

      Sonal: Right. And you’re saying that point of distinction comes from how you tie together that technology process and culture through DevOps.

      Jez: Right. And it can provide a competitive advantage to your business. If you’re buying something that everyone else also has access to, then it’s no longer a differentiator. But if you have an in-house capability, and those people are finding ways to drive your business — I mean, this is the classic Amazon model, they’re running hundreds of experiments in production at any one time to improve their product. And that’s not something that anyone else can copy. That’s why Amazon keeps winning. So what people are doing is copying the capability instead. And that’s what we’re talking about. How do you build that capability?

      Organizational types

      Sonal: The most fascinating thing to me about all this is honestly not the technology, per se, but the organizational change part of it, and the organizations themselves. So of all the people you studied, is there an ideal organizational makeup that is ideal for DevOps? Or is it one of these magical formulas that has this ability to turn a big company into a startup and a small company into — because that’s actually the real question.

      Dr. Forsgren: From what I’ve seen, there might be two ideals. The nice happy answer is the ideal organization is the one that wants to change.

      Sonal: I mean, given this huge n=23,000 data set, is it not tied to a particular profile of a size of company? They’re both shaking their heads, just for the listeners.

      Dr. Forsgren: I see high performers among large companies. I see high performers in small companies. I see low performers in small companies. I see low performers in highly regulated companies. I see low performers in non-regulated companies…

      Sonal: So, tell me the answer you’re not supposed to say.

      Dr. Forsgren: So, that answer is, it tends to be companies that are like, “Oh, shit.” And there are two profiles — either, one, they’re like way behind, and “Oh, shit,” and they have some kind of funds. Or, they are, like, this lovely, wonderful bastion of like — they’re these really innovative high performing companies. But they still realize they’re a handful of like two or three companies ahead of them. And they don’t wanna be number two, they are gonna be number one.

      Sonal: So, those are the ideal — I mean, just to, like, anthropomorphize it a little bit. It’s like the 35- to 40-year-old who suddenly discovers you might be pre-diabetic, so you better do something about it now before it’s too late. But it’s not too late, because you’re not so old, where you’re about to reach sort of the end of a possibility to change that runway. And then there’s this person who’s, sort of, kind of already like in the game running in the race, and there might be two or three but they want to be like, number one.

      Jez: And I think to extend your metaphor, the companies that do well are the companies that never got diabetic in the first place because they always just ate healthily like…

      Sonal: They were already glucose monitoring. They had continuous glucose monitors on, which is like DevOps, actually.

      Dr. Forsgren: They were always athletes.

      Jez: Right. You know, diets are terrible because at some point, you have to stop the diet.

      Sonal: And it has a sudden start and stop, as opposed to a way of life, is what you’re saying.

      Jez: Right, exactly. So, if you just always eat healthily and never eat too much, or very rarely eat too much and do a bit of exercise every day, you never get to the stage where, “Oh, my God, now I can only eat tofu.”

      Dr. Forsgren: Yeah, totally. So, like, my “loving professorness nurture Nicole” also has one more profile that, like, I love, and I worry about them, like mother hen. And it’s the companies that I talk to, and they come to me and they’re struggling. And I haven’t decided if they wanna change, but they’re like, “So we need to do this transformation. And we’re gonna do the transformation.” And it’s either because they want to, or they’ve been told that they need to. And then they will insert this thing where they say, “But I’m not a technology company.” I’m like, but we just had this 20-minute conversation about how you’re leveraging technology to drive value to customers, or to drive this massive process that you do. <Yeah.> And then they say, “But I’m not a technology company.” I could almost see why they had that in their head because they were a natural resources company. But there was another one where they were a finance company.

      Sonal: I mean, an extension of “software eats the world” is really every company is a technology company. It’s fascinating to me that that third type exists, but it is a sign of this legacy world moving into software.

      Dr. Forsgren: Right. And I worry about them. Also, at least for me, personally, you know. I’ve lived through this, like, mass extinction of several firms, and I don’t want it to happen again. <Yeah.> And I worry about so many companies that keep insisting they’re not technology companies, and I’m like, “Oh, honey child…”

      Sonal: You’re a tech company.

      Jez: You know, one of the gaps in our data is actually China. And I think China is a really interesting example because they didn’t go through the whole, you know, “IT doesn’t matter” phase, they’re jumping straight from no technology to Alibaba and Tencent, right? I think U.S. companies should be scared, because at the moment, Tencent and Alibaba are already moving into other developing markets, and they are gonna be incredibly competitive because it’s just built into their DNA.

      Productivity and performance

      Sonal: So, the other fascinating thing to me is that you, essentially, were able to measure performance of software, and clearly productivity. [Are] there any more insights on the productivity side?

      Jez: Yes, yes, I wanna go.

      Dr. Forsgren: This is his favorite rant.

      Sonal: For everybody else, he’s like jumping around and, like, waving his hands. So tell us!

      Jez: The reason the manufacturing metaphor breaks down is because in manufacturing, you have inventory.

      Sonal: Yes.

      Jez: We do not have inventory in the same way in software. In a factory, like the first thing your Lean consultant is gonna do, walking into the factory is point to the piles of thing everywhere. But I think if you walk into an office where there’s developers, where’s the inventory?

      Dr. Forsgren: By the way, that’s what makes talking about this to executives so difficult — they can’t see the process.

      Jez: Well, it’s a hard question to answer, because is the inventory the code that’s being written? And people actually have done that and said, “Well, listen, lines of code are an accounting measure and we’re going to capture that as, you know, capital, which we’re gonna…

      Sonal: That’s insane, it seems like an invitation to write crappy, unnecessarily long code.

      Jez: That’s exactly what happens and then…

      Sonal: It’s like the olden days of getting paid for a book by how long it is. And it’s like, actually really boring when you can actually write it in like one third of the length.

      Dr. Forsgren: Let’s write it in German.

      Jez: Right, you know…

      Sonal: <laughter> I was thinking of Charles Dickens.

      Jez: In general, you know, you prefer people to write short programs, because they’re easier to maintain, and so forth. But lines of code have all these drawbacks, we can’t use them as a measure of productivity.

      Sonal: So if you can’t measure lines of code, what can you measure? Because I really want an answer. Like how do you measure productivity?

      Jez: So velocity is the other classic example. Agile, there’s this concept of velocity, which is the number of story points a team manages to complete in an iteration. So before the start of an iteration, in many Agile — particularly Scrum-based processes, you’ve got all this work to do, like, we need to build these five features, how long will this feature take? And the developers fight over it, and they’re like, “Oh, it’s five points.” And then this one’s gonna take three points, this one’s gonna take two points. And so you have a list of all these features, you don’t get through all of them. But at the end of the iteration, the customer signs off, “Well, I’m accepting this one, this one’s fine, this one’s fine, this one’s a hot mess, go back and do it again,” whatever. The number of points you complete in the iteration is the velocity.

      Sonal: So it’s like the speed at which you’re able to deliver those features.

      Jez: So, a lot of people treat it like that. But actually that’s not really what it’s about. It’s a relative measure of effort. And it’s for capacity planning purposes. So you basically, for the next iteration, will only commit to completing the same velocity that we finished last time, so it’s relative, and it’s team dependent. And so what a lot of people do is they start comparing velocities across teams. Then what happens is, a lot of work you need to collaborate between teams. But hey, if I’m gonna help you with your story, that means I’m not gonna get my story points and you’re gonna get your story points.

      Sonal: So, it’s, like, bad incentive structure, right?

      Jez: Right, people can game it as well. You should never use story points as a productivity measurement.

      Sonal: So, lines of code doesn’t work, velocity doesn’t work — what works?

      Jez: So, this is why we like two things, in particular — one thing that it’s a global measure, and secondly, that it’s not just one thing, it mixes two things together, which might normally be in tension. And so this is why we went for our measure of performance. So measuring lead time, release frequency, and then time for a store and change fail rate. Lead time is really interesting because lead time is on the way to production, right? So all the teams have to collaborate — it’s not something where, you know, I can go really fast in my velocity, but nothing ever gets delivered to the customer. That doesn’t count in lead time, so it’s a global measure.

      Sonal: So, it takes care of that problem of the incentive alignment around the competitive dynamic.

      Jez: Exactly.

      Dr. Forsgren: Also, it’s an outcome, it’s not an output.

      Jez: There’s a guy called Jeff Patton. He’s a really smart thinker in the kind of Lean-Agile space. He says, “Minimize output, maximize outcomes,” which I think is simple but brilliant.

      Sonal: It’s so simple because it just shifts the words to impact.

      Jez: And even we don’t get all the way there because we’re not yet measuring, did the features deliver the expected value to the organization to the customers?

      Dr. Forsgren: Well, we do get there because we focus on speed and stability, <Mmhmm> which then deliver the outcome to the organization, profitability, productivity, market share.

      Sonal: But the second half of this, which I am also hearing is, did it meet your expectations? Did it perform to the level that you wanted it to? Did it match what you asked for? Or even if it wasn’t something you specified that you desired or needed? That seems like a slightly open question?

      Jez: So, we did actually measure that. We looked at nonprofit organizations, and these were exactly the questions we measured. We asked people, did the software meet — I can’t remember what the exact questions were.

      Dr. Forsgren: Effectiveness, efficiency, customer satisfaction, delivering mission goals…

      Sonal: How fascinating that you do it nonprofits, because that is a larger movement along nonprofit measurement space, to try to measure impact?

      Dr. Forsgren: But we captured it everywhere <Yeah.> because even profit-seeking firms still have these goals.

      Jez: Yeah, in fact, as we know, from research, companies that don’t have a mission, other than making money, do less well than the ones that do. But I think, again, what the data shows is that companies that do well on the performance measures we talked about outperform their low-performing peers by a factor of two. Our hypothesis is, what we’re doing when we create these high-performing organizations, in terms of speed and stability, is we’re creating feedback loops. What it allows us to do is build a thin slice, a prototype of a feature, get feedback through some UX mechanism, whether that’s showing people with a prototype and getting their feedback, whether it’s running A/B tests or multivariate tests in production. It’s what creates these feedback loops that allow you to shift direction very fast.

      Sonal: I mean, that is at the heart of Lean startup, it’s the heart of anything you’re putting out into the world — is you have to kind of bring it full circle. It is a secret of success to Amazon, as you cited earlier. I would distill it to just that. I think I heard Jeff Bezos say the best line. It was at the Internet Association dinner in DC last year, where someone asked about an innovation. He’s like, to him an innovation is something that people actually use. <Right.> And that’s what I love about the feedback loop thing is it actually reinforces that mindset of that’s what innovation is.

      Jez: Right. So to sum up, the way you can frame this is, DevOps is that technological capability that underpins your ability to practice Lean startup and all these very rapid iterative processes.

      Sonal: So I have a couple of questions then. So one is, you know, going back to this original taxonomy question, and you guys describe that there isn’t necessarily an ideal organizational type…

      Dr. Forsgren: Which, by the way, should be encouraging.

      Sonal: I agree! I think it’s super encouraging and more importantly, democratizing, that anybody can become a hit player.

      Jez: We were doing this in the federal government.

      Sonal: I love that. But one of my questions is, when — we had Adrian Cockcroft, on this podcast a couple of years ago, talking about microservices. And the thing that I thought was so liberating about what he was describing the Netflix story, was that it was a way for teams to essentially become little mini product management units, and essentially self-organize. Because the infrastructure, by being broken down into these micro pieces, versus, say, a monolithic kind of uniform architecture — I would think that being a, you know, organization that’s containerized its code in that way, that has this microservices architecture, would be more suited to DevOps — or is that a wrong belief? I’m just trying to understand, again, that taxonomy thing of how these pieces all fit together.

      Jez: So we actually studied this. There’s a whole section on architecture in the book <Oh, great!> where we looked at exactly this question. Architecture has been studied for a long time, and people talk about architectural characteristics. There’s the ATAM, the architectural trade-off model that Carnegie Mellon developed. There’s some additional things we have to care about — testability and deployability. Can my team test its stuff without having to rely on this very complex integrated environment? Can my team deploy its code to production without these very complex orchestrated deployments? Basically, can we do things without dependencies? That is one of the biggest predictors, in our cohort of IT performance, is the ability of teams to get stuff done on their own without dependencies on other teams, whether that’s testing or whether it’s deploying or is planning…

      Dr. Forsgren: Or, even just communicating. <Yeah.> Can you get things done without having to do, like, mass communication and checking and permissions.

      Sonal: Question I love love love asking on this podcast is, we always revisit the 1937 Coase paper about the theory of the firm <Yes!> and this idea that transaction costs are more efficient. And this is, like, the ultimate model for reducing friction <Yeah.> and those transaction costs, communication coordination costs, all of it.

      Jez: That’s what — like all the technical and process stuff is about that. I mean, Don Reinertsen, once came to one of my talks on continuous delivery, at the end, he said, “So continuous delivery, that’s just about reducing transaction costs, right?” And I’m like…

      Sonal: Huh, an economist’s view of DevOps. I love it.

      Jez: You’re right. You reduced my entire body of words to one sentence.

      Dr. Forsgren: It’s so much Conway’s Law, right?

      Sonal: Remind me what Conway’s Law is.

      Dr. Forsgren: So organizations which design systems are constrained to produce designs, which are copies of the communication structures of these organizations.

      Sonal: Oh, right, it’s that idea, basically, that your software code looks like the shape of the organization itself.

      Jez: Right.

      Dr. Forsgren: And how we communicate, right? So which, you know, Jez just summarized, if you have to be communicating and coordinating with all of these other different groups…

      Sonal: Command and control looks like waterfall, a more decentralized model looks like independent teams.

      Jez: Right. So the data shows that. One thing that I would say is, a lot of people jump on the microservices containerization bandwagon, there’s one thing that is very important to bear in mind. Implementing those technologies does not give you those outcomes we talked about. We actually looked at people doing mainframe stuff — you can achieve these results with mainframes. Equally, you can use, you know, Kubernetes and, you know, Docker and microservices and not achieve these outcomes.

      Dr. Forsgren: We see no statistical correlation with performance, whether you’re on a mainframe, or a Greenfield or Brownfield system, if you’re building something brand new, or if you’re working on an existing build. And one thing I wanted to bring up that we didn’t before, is I said, you know, day one is short, day two is long. And I talked about things that live on the internet and live on the web. <Yeah.> This is still a really, really smart approach for packaged software. And I know people who are working in and running packaged software companies that use this methodology because it allows them to still work in small, fast approaches. And all they do is they push to a small package, pre-production database, and then when it’s time to push that code onto some media, they do that.

      Determining readiness to change

      Sonal: Okay. So what I love hearing about this is that it’s actually not necessarily tied, again, to the architecture or the type of company you are, that there’s opportunity for everybody. But there is this mindset of, like, an organization that is ready, it’s like a readiness level for a company.

      Dr. Forsgren: Oh, I hear that all the time. I don’t know if I’d say there’s any such thing as readiness, right? Like, there’s always an opportunity to get better, there’s always an opportunity to transform. The other thing that really, like, drives me crazy and makes my head explode is this whole maturity model thing. Okay, are you ready to start transforming? Well, like, you can just not transform and then maybe fail, right? Maturity models, they’re really popular in industry right now, but I really can’t stress enough that they’re not really an appropriate way to think about technology transformation.

      Sonal: I was thinking of readiness in a kind of, like, NASA Technology Readiness Levels, or TRLs, which is something we used to think about a lot for very early-stage things. But you’re describing maturity of an organization, and it sounds like there’s some kind of a framework for assessing the maturity of an organization. And you’re saying that doesn’t work. But first of all, what is that framework? And why doesn’t it work?

      Dr. Forsgren: Well, so many people think that they want a snapshot of their like DevOps or their technology transformation, and spit back a number, right? And then you will have one number to compare yourself against everything. The challenge, though, is that a maturity model usually is leveraged to help you think about arriving somewhere. And then here’s the problem — once you’ve arrived, what happens?

      Jez: Oh, we’re done.

      Dr. Forsgren: You’re done! And then the resources are gone. And by resources, I don’t just mean money, I mean time, I mean attention. We see, year over year over year, the best, most innovative companies continue to push. So what happens when you’ve “arrived,” I’m using my finger quotes.

      Sonal: You stop pushing.

      Dr. Forsgren: You stop pushing. What happens when executives or leaders or whomever decide that you no longer need resources of any type?

      Sonal: I have to push back again, though. Doesn’t this help — because it is helpful to give executives in particular, particularly those that are not tech native, coming from the seeds of the engineering organization, some kind of metric to put your head around. “Where are we, where are we at?”

      Dr. Forsgren: So you can use a capability model. <Mmm.> You can think about the capabilities that are necessary to drive your ability to develop and deliver software with speed and stability. Another limitation is that they’re often kind of a lockstep or a linear formula, right?

      Sonal: No, right. It’s like a stepwise A, B, C, D, E, 1, 2, 3, 4. And in fact, the very nature of anything iterative is its very nonlinear and circular. Feedback loops are circles.

      Dr. Forsgren: Right. Maturity models just don’t allow that. Now another thing that’s really, really nice is that capability models allow us to think about capabilities in terms of these outcomes. Capabilities drive impact. Maturity models are just this thing where you have this level one, level two, level three, level four.

      Sonal: That’s a bit performative.

      Dr. Forsgren: And then, finally, maturity models just sort of take this snapshot of the world and describe it. How fast is technology and business changing? If we create a maturity model now, let’s wait — let’s say, four years — that maturity model is old and dead and dusty and gone.

      Sonal: Do new technologies change the way you think about this? Because I’ve been thinking a lot about how product management for certain types of technologies changes with the technology itself, and that machine learning and deep learning might be a different beast. And I’m just wondering if you guys have any thoughts on that.

      Jez: Yeah. I mean, me and Dave Farley wrote the “Continuous Delivery” book back in 2010. And since then, you know, there’s Docker and Kubernetes and large-scale launch from the cloud, and all these things that you had no idea would happen. People sometimes ask me, you know, isn’t it time you wrote a new edition of the book? I mean, yeah, we could probably rewrite it. Does it change any of the fundamental principles? <Mmhmm.> No. Do these new tools allow you to achieve those principles in new ways? Yes.

      So I think, you know, this is how I always come back to any problem is, go back to first principles. The first principles, I mean, they will change over the course of centuries. I mean, we’ve got modern management versus kind of scientific management, but they don’t change over the course of, like, a couple of years. The principles are still the same, technologies give you new ways to do them, and that’s what’s interesting about them.

      Equally, things can go backwards. A great example of this is — one of the capabilities we talk about in the book is working off a shared trunk, or master, in version control, not going on these long-lived feature branches. And the reason for that is actually because of feedback loops. You know, developers love going off into a corner, putting headphones on their head, <Yeah.> and coding something for, like, days. And then they try to integrate it into trunk, you know, and that’s a total nightmare. And not just for them, more critically, for everyone else, who then has to merge their code into whatever they’re working on. So that’s hugely painful. Git is one of these examples of a tool that makes it very easy for people, like, “Oh, I can use feature branches.” So I think, again, it’s nonlinear in the way that you describe, gives you new ways to do things. Are they good and bad? It depends.

      Sonal: But the thing that strikes me about what you guys have been talking about, as a theme in this podcast, that seems to lend itself well to the world of machine learning and deep learning, where that technology might be different, is it sort of lends itself to a probabilistic way of thinking, and that things are not necessarily always complete. <Yes, absolutely.> And that there is not a beginning and an end, and that you can actually live very comfortably in an environment where things are, by nature, complex, and that complexity is not necessarily something to avoid. So in that sense, I do think there might be something kind of neat about ML and deep learning and AI, for that matter, because it is very much lending itself to that sort of mindset.

      Jez: Yeah. And in our research, we talk about working in small batches. <Mmhmm.> There’s a great video by Bret Victor called “Inventing on Principle,” where he talks about how important it is to the creative process to be able to see what you’re doing. And he has this great demo of this game he’s building where he can change the code, and the game changes its behavior instantly. When you’re doing things like that…

      Sonal: You don’t get to see that.

      Jez: No. And the whole thing with machine learning is, how can we get the shortest possible feedback from changing the input parameters, to seeing the effects so that the machine can learn? And that the moment you have very long feedback loops, the ML becomes much much harder because you don’t know which of the input changes caused the change in output that the machine is supposed to be learning from. So the same thing is true of organizational change and process, and product development as well, by the way, which is working in small batches, so that you can actually reason about cause and effects. You know, I changed this thing, it had this effect. Again, that requires short feedback loops, that requires small batches. That’s one of the key capabilities we talk about in the book. And that’s what DevOps enables.

      Getting started and company culture

      Sonal: So we’ve been in this hallway-style conversation around all these themes of DevOps, measuring it, why it matters, and what it means for organizations. But practically speaking, if a company — and you guys are basically arguing it, any company — not necessarily a “company that thinks it’s a tech company,” and necessarily a company that has, like, this amazing, modern infrastructure stack — it could be a company that’s still working off mainframes. What should people actually do to get started, and how do they know where they are?

      Dr. Forsgren: So what you need to do is, take a look at your capabilities, understand what’s holding you back, right? Try to figure out what your constraints are. <Mmhmm.> But the thing that I love about much of this is, you can start somewhere, and culture is such a core important piece. We’ve seen across so many industries, culture is truly transformative.

      Jez: And in fact, we measure it in our work and we can show that culture has a predictive effect on organizational outcomes and on technology capabilities. We use a model from a guy called Ron Westrum, who was a social scientist studying safety outcomes, in fact, in safety-critical industries like healthcare and aviation. He created a typology where he organizes organizations based on whether they’re pathological, bureaucratic, or generative.

      Sonal: That’s actually a great typology. I want to apply that to people I date. <laughter>

      Dr. Forsgren: I know, right? Too real.

      Sonal: I want to apply that to people. <laughter>

      Jez: There’s a book in there, definitely.

      Sonal: I like how I’m trying to anthropomorphize all these organizational things into people. But anyway, go on Jez.

      Jez: Yeah, so instead of the five love languages, we can have the three relationship types. So pathological organizations are characterized by low cooperation between different departments and up and down the organizational hierarchy. How do we deal with people who bring us bad news? Do we ignore them, or do we shoot people who bring us bad news? How do we deal with responsibilities? Are they defined tightly so that when something goes wrong, we know whose fault it is so we punish them? Or do we share risks? Because we know we’re all in this together and it’s the team…

      Sonal: You all have skin in the game, you’re all accountable, right.

      Jez: Exactly. How do we do with bridging between different departments? And crucially, how do we deal with failure? As we discussed earlier, in any complex system, including organizational systems, failure is inevitable. So failure should be treated as a learning opportunity. Not whose fault was it, but why did that person not have the information they needed, the tools they needed? How can we make sure that when someone does something, it doesn’t lead to catastrophic outcomes, but instead, it leads to contained small blast radiuses?

      Sonal: Right, not an outage on Black Friday.

      Jez: Right, exactly. And then also, how do we deal with novelties? Is novelty crushed or is it implemented or does it lead to problems? One of the pieces of research that kind of confirms what we were talking about was some research that was done by Google. They were trying to find what makes the greatest Google team — you know, is it four Stanford graduates, and no developer, and fire all the managers…

      Sonal: Right.

      Dr. Forsgren: Is it a data scientist, an OGS programmer, and a manager…

      Sonal: Right, one product manager paired with one system engineer, with one…

      Dr. Forsgren: Yep.

      Jez: And what they found was the number one ingredient was psychological safety. <Ah, interesting.> Does the team feel safe to take risks? And this ties together failure and novelty. If people don’t feel that when things go wrong, they’re gonna be supported, they’re not gonna take risks, and then you’re not gonna get any novelty because novelty, by definition, involves taking risks. So we see that one of the biggest things you can do is create teams where it’s safe to go wrong and make mistakes, and then where people will treat that as a learning experience. This is a principle that applies, again, not just in product development, you know, the Lean startup, “fail early, fail often,” but also in the way we deal with problems at an operational level as well.

      Dr. Forsgren: And how we interact with our team when these things happen.

      Sonal: To just to kind of summarize that, you have pathological, this is a power-oriented thing where, you know, the people are scared, the messenger is gonna be shot. Then you have this bureaucratic kind of rule-oriented world where the messengers aren’t heard. And then you have this sort of generative — and again, I really wish I could apply this to people, but we’re talking about organizations here — for culture, which is more performance-oriented.

      Jez: And I just wanna add one thing about this, you know, working in the federal government, you would imagine that to be a very bureaucratic organization.

      Sonal: I would, actually.

      Jez: And actually, what was surprising to me was that, yes, there’s lots of rules — the rules aren’t necessarily bad, that’s how we can operate at scale, is by having rules. But what I found was, there [were] a lot of people who are mission-oriented. And I think that’s a nice alternative way to think about generative organizations is to think about mission orientation. The rules are there, but if it’s important to the mission, we’ll break the rules.

      Dr. Forsgren: And we measure this at the team level, right? Because you can be in the government, and there were pockets that were very generative. <Right.> You can be in a startup <Yep.> and you can see startups that act very bureaucratic.

      Jez: Or pathological, even.

      Dr. Forsgren: Or very pathological.

      Sonal: Right, and that’s like the people where…

      Jez: The cult of the CEO…

      Sonal: …where it’s not charismatic, inspirational vision, but to the expense of actually being heard and the messenger is shot, etc.

      Dr. Forsgren: And we have several companies around the world now that are measuring their culture on a quarterly cadence basis, because we show in the book how to measure it. Westrum’s typology was the table itself, and so we turned that into a scientific, psychometric way to measure it.

      Sonal: Now, this makes sense why I’m putting these anthropomorphic analogies because, in this sense, organizations are like people.

      Dr. Forsgren: They’re made of people.

      Sonal: Teams are organic entities. And I love that you said that the unit of analysis is a team because it means you can actually do something. You can start there and then you can like, see if it actually spreads or doesn’t spread, bridges, doesn’t bridge, etc. And what I also love about this framework is, it also moves away from this cult of failure mindset that I think people tend to have, where it’s like failing for the sake of failing. And you actually wanna avoid failure. <Right.> And the whole point of failing is to actually learn something and then be better and take risks so you can implement this new phase.

      Dr. Forsgren: And very smart risks.

      Final thoughts and recommendations

      Sonal: So what’s your final — I mean, there’s a lot of really great things here, but, like, what’s your final sort of parting takeaway for listeners or people who might wanna get started or think about how they are doing?

      Jez: So I think, you know, we’re in a world where technology matters. Anyone can do this stuff, but you have to get the technology part of it right. That means investing in your engineering capabilities, in your process, in your culture, in your architecture. We’ve dealt with a lot of things here that people think are intangible, and we’re here to tell you, they’re not intangible. You can measure them, they will impact the performance of your organization. So take a scientific approach to improving your organization and you will reap the dividends.

      Sonal: When you guys talk about, you know, anyone can do this, the teams can do this — but what role in the organization is usually most empowered to be the owner of where to get started? Is it like the VP of engineering, is it the CTO, the CIO?

      Dr. Forsgren: I was gonna say, don’t minimize the role of and the importance of leadership. DevOps sort of started as a grassroots movement. But right now we’re seeing roles like VP and CTO being really impactful, in part because they can set the vision for an organization but also in part because they have resources that they can dedicate to this.

      Sonal: We see a lot of CEOs and CTOs and CIOs in our business. We have like a whole briefing center, we hear what’s top of mind for them all the time. Everyone thinks they’re transformational. So like, what actually makes a visionary type of leader who has that, not just the purse strings and the decision-making power, but the actual characteristics that are right for this?

      Dr. Forsgren: Right. And that’s such a great question. And so we actually dug into that in our research, and we find that there are five characteristics that end up being predictive of driving change and really amplifying all of the other capabilities that we found. And these five characteristics are vision, intellectual stimulation, inspirational communication, supportive leadership, and personal recognition. And so, what we end up recommending to organizations is, absolutely invest in the technology. Also, invest in leadership in your people because that can really help drive your transformation home.

      Sonal: Well, Nicole, Jez, thank you for joining the “a16z Podcast.” The book, just out, is “Accelerate: Building and Scaling High Performing Technology Organizations.” Thank you so much, you guys.

      Jez: Thanks for having us.

      Dr. Forsgren: Thank you.

      • Nicole Forsgren is the VP, Research & Strategy at GitHub. Author of the award-winning book “Accelerate: The Science of Lean Software and DevOps”, she was also lead investigator on the largest DevOps studies to date.

      • Jez Humble

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Why We Shouldn’t Fear the ‘Black Box’ of AI (in Healthcare and Everywhere)

      Vijay Pande

      Alongside the excitement and hype about our growing reliance on artificial intelligence, there’s intense fear about the way the technology works. A 2017 MIT Technology Review article titled “The Dark Secret at the Heart of AI” warned, “No one really knows how the most advanced algorithms do what they do. That could be a problem.” Thanks to this uncertainty and lack of accountability, a report by the AI Now Institute at NYU recommended that public agencies responsible for criminal justice, health care, welfare, and education shouldn’t use such technology.

      Given these types of concerns, the unseeable space between where data goes in and answers come out is often referred to as a “black box” — seemingly a reference to the hardy (and in fact orange, not black) data recorders mandated on aircraft and often examined after accidents. In the context of AI, the term more broadly suggests an image of being in the “dark” about how the technology works: We put in and provide the data and models and architectures, and then computers provide us answers while continuing to learn on their own, in a way that’s seemingly impossible — and certainly too complicated — for us to understand.

      There’s particular concern about this in health care, where AI is used to classify which skin lesions are cancerous, to identify very early stage cancer from blood, to predict heart disease, to determine what compounds in people and animals could extend healthy life spans, and more. But these fears about the implications of black box are misplaced. AI is no less transparent than the way in which doctors have always worked — and in many cases it represents an improvement, augmenting what hospitals can then do for patients and the entire health care system. After all, the black box in AI isn’t a new problem due to new tech: Human intelligence itself is — and always has been — a black box.

      Let’s take the example of a human doctor making a diagnosis. Afterward, a patient might ask that doctor how she made that diagnosis, and she would probably share some of the data she used to draw her conclusion. But could she really explain how and why she made that decision, what specific data from what studies she drew on, what observations from her training or mentors influenced her, what tacit knowledge she gleaned from her own and her colleagues’ shared experiences, and how all of this combined into that precise insight? Sure, she’d probably give us a few indicators about what pointed her in a certain direction — but there would also be an element of guessing, of following hunches. And even if there weren’t, we still wouldn’t know that there weren’t other factors involved, of which she wasn’t even consciously aware.

      If the same diagnosis had been made with AI, we could draw from all available information on that particular patient — as well as data anonymously aggregated across time and from countless other relevant patients everywhere, in order to make the strongest evidence-based decision possible. It would be a diagnosis with a direct connection to the data, rather than human intuition based on limited data and derivative summaries of anecdotal experiences with a relatively small number of local patients.

      But we make decisions in areas that we don’t fully understand every day — often very successfully — from the predicted economic impacts of policies and weather forecasts to how we conduct much of science in the first place. We either oversimplify things or accept that they’re too complex for us to break down linearly, let alone explain fully. It’s just like the black box of AI: human intelligence can reason and make arguments for a given conclusion, but it can’t explain the complex, underlying basis for how we arrived at a particular conclusion. Think of what happens when a couple gets divorced because of one stated cause — “infidelity” — when in reality there’s an entire unseen universe of intertwined causes, forces, and events that contributed to that outcome. Why did they choose to split up when another couple in a similar situation didn’t? Even those inside of it can’t fully explain it. It’s a black box.

      The irony is that compared to human intelligence, AI is actually the more transparent of intelligences! Unlike the human mind, AI can — and should — be interrogated and interpreted. From the ability to audit and refine models and expose knowledge gaps in deep neural nets to the debugging tools that will inevitably be built and the potential ability to augment human intelligence via brain-computer interfaces, there are many technologies that could help interpret artificial intelligence in a way we can’t do in interpreting the human brain. In the process, we may even learn more about how human intelligence itself works.

      Perhaps the real source of critics’ concerns isn’t that we can’t “see” AI’s reasoning — it’s that as AI gets more powerful, the human mind becomes the limiting factor. It’s that, in the future, we’ll basically need AI to understand AI. In health care as well as in other fields, this means we will soon see the creation of a new category of human professionals who don’t have to make the moment-to-moment decisions themselves, but instead manage a team of AI workers — just like commercial airplane pilots who engage autopilots to land in poor weather conditions. Doctors will no longer “drive” the primary diagnosis; instead, they’ll ensure that the diagnosis is relevant and meaningful for a patient, and oversee when and how to offer more clarification and more narrative explanations. The doctor’s office of the future will very likely include computer assistants, on both the doctor’s side and the patient’s side, as well as data inputs that come from far beyond the office walls.

      When this happens, it will become clear that the so-called “black box” of AI is more of a feature, not a bug — because it’s more possible to capture and explain what’s going on there than it is in the human mind. None of this dismisses or ignores the need for AI oversight. It’s just that instead of worrying about the black box, we should focus on the opportunity — and therefore better address a future — where AI not only augments human intelligence and intuition, but perhaps even sheds light on and redefines what it means to be human in the first place.

      This op-ed originally appeared in The New York Times. 

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      Putting AI in Medicine, in Practice

      Brandon Ballinger, Vijay Pande, Mintu Turakhia, and Hanne Winarsky

      There’s been a lot of talk about technology — and AI, deep learning, and machine learning specifically — finally reaching the healthcare sector. But AI in medicine isn’t actually new; it’s actually been there since the 1960s. And yet we didn’t see it effect a true change, or even become a real part our doctor’s offices — let alone routine healthcare services. So: what’s different now? And what does AI in medicine look like, practically speaking, whether it’s ensuring the best data, versioning software for healthcare, or other aspects?

      In this episode of the a16z Podcast, Brandon Ballinger, CEO of Cardiogram; Mintu Turakhia, cardiologist at Stanford and Director of the Center for Digital Health; and general partner and head of a16z bio fund Vijay Pande in conversation with Hanne Winarksy discuss where will we start to see AI in healthcare first — diagnosis, treatment, or system management — to what it will take for it to succeed. Will we perhaps see a “levels” of AI framework for doctors as we have for autonomous cars?

      Show Notes

      • Discussion of how AI has been used so far in medicine [0:45] and its potential for the future [3:55]
      • Questions about data sets [7:46] and how AI might be scaled [12:45]
      • Unknowns about integrating AI into medicine [15:01], and a discussion of incentives for making the transition [21:04]

      Transcript

      Hanne: Hi, and welcome to the “a16z Podcast.” I’m Hanne, and today we’re talking about AI in medicine. But we want to talk about it in a really practical way. What it means to use it in practice and in a medical practice, what it means to build medical tools with it — but also what creates the conditions for AI to really succeed in medicine, and how we design for those conditions, both from the medical side and from the software side. 

      The guests joining for this conversation, in the order in which you will hear their voices, are Mintu Turakhia, a cardiologist at Stanford and director of the Center For Digital Health; Brandon Ballinger, CEO and founder of Cardiogram, a company that uses heart rate data to predict and prevent heart disease; and Vijay Pande, a general partner here at a16z and head of our bio fund. 

      So, let’s maybe just do a quick breakdown of what we’re actually talking about when we talk about introducing AI to medicine. What does that actually mean? How will we actually start to see AI intervene in medicine and in hospitals and in waiting rooms?

      AI’s history and its future potential

      Dr. Turakhia: AI is not new to medicine. Automated systems in healthcare have been described since the 1960s. And they went through various iterations of expert systems and neural networks and [were] called many different things.

      Hanne: In what way would those show up in the ’60s and ’70s?

      Dr. Turakhia: So, at that time, there were no high resolutions. There weren’t too many sensors. And it was about a synthetic brain that could take what a patient describes as the inputs, and what a doctor finds on the exam as the inputs.

      Hanne: Using verbal descriptions?

      Dr. Turakhia: Yeah. Basically words. People created what are called ontologies and classification structures. You put in the ten things you felt, and a computer would spit out the top ten diagnoses in order of probability. And even back then, they were outperforming average physicians. So, this is not a new concept.

      Hanne: So, basically doing what hypochondriacs do with Google today, but verbally.

      Dr. Turakhia: Right. So, Google is, in some ways, an AI expression of that, where it’s actually used ongoing inputs and classification to do that over time. [It’s a] much more robust neural network, so to speak.

      Brandon: So, an interesting case study is the MYCIN system, which is from 1978, I believe. And so, this was an expert system trained at Stanford. It would take inputs that were just typed in manually, and then it would essentially try to predict what a pathologist would show. And it was put to the test against five pathologists, and it beat all five of them.

      Hanne: And it was already outperforming.

      Brandon: It was already outperforming doctors, but when you go to the hospital, they don’t use MYCIN or anything similar. And I think this illustrates that sometimes, the challenge isn’t just the technical aspects or the accuracy, it’s the deployment path. Some of the issues around there are — okay, is there a convenient way to deploy this to actual physicians? Who takes the risk? What’s the financial model for reimbursement? And so, if you look at the way the financial incentives work, there are some things that are backwards. For example, if you think about a hospital from the CFO’s perspective, a misdiagnosis actually earns them more money…

      Hanne: What?

      Brandon: …because when you misdiagnose, you do follow-up tests, right? And those — and our billing system is fee-for-service. So, every little test that’s done is billed for.

      Hanne: But nobody wants to be giving out wrong diagnoses, so where’s the incentive? The incentive is just in the system — the money that results from it.

      Brandon: No one wants to give an incorrect diagnosis. On the other hand, there’s no budget to invest in better diagnoses.

      Hanne: In making sure it doesn’t happen.

      Brandon: I think that’s been part of the problem. And so, things like fee-for-value are interesting because now, you’re paying people for an accurate diagnosis or for a reduction in hospitalizations, depending on the exact system. And so, I think that’s the case where accuracy is rewarded with a greater payment, which sets up the incentives so that AI can actually win in this circumstance.

      Dr. Turakhia: Where I think AI has come back at us with a force is — it came to healthcare as a hammer looking for a nail. What we’re trying to figure out is where you can implement it easily and safely, with not too much friction and with not a lot of physicians going crazy, and where it’s going to be very, very hard. And that, I think, is the challenge in terms of building, developing these technologies, commercializing them, and seeing how they scale. And so, the use cases really vary across that spectrum.

      Brandon: Yeah, I think about there as being a couple different cases where AI can intervene. One is to substitute what doctors do already, and so people use the example of radiology as an example. The other area that I think is maybe more interesting is that AI can complement what doctors can’t do already.

      So, it would be possible for a doctor to, say, read an ECG and tell you whether you’re in an abnormal heart rhythm. No doctor right now can read your Fitbit data and tell you whether you have a condition like sleep apnea. I mean, if you look at your own data, you can kind of see restful sleep as real structured REM cycles, so you can see some patterns there. That said, the gold standard that a sleep doctor would use is a sleep study, where they wire you up with six different sensors and tell you to sleep naturally. There’s a big difference here between the very noisy consumer sensors that may be less interpretable, and what a doctor is used to seeing.

      Vijay: Or it could be that the data is on the device, but the analysis can’t be done yet. Maybe the analysis needs a gold standard data set to compare to. There are a lot of missing parts beyond just gathering the data from the patient in the first place.

      Dr. Turakhia: I think there’s some inherent challenges in the nature of the beast. Healthcare is unpredictable. It’s stochastic. You can predict a cumulative probability — like a probability of getting condition X, or diagnosis X, over a time horizon of 5 or 10 years — but we are nowhere near saying, “You’re going to have a heart attack in the next 3 days.” Prediction is very, very, very difficult, and so where prediction might have a place is where you’re getting high fidelity data, whether it’s from a wearable or a sensor. 

      It’s so dense that a human can’t possibly do it. Like, a doctor’s not going to look at it. And two, it’s relatively noisy. Inaccurate, poor classifiers, missing — periods where you don’t have this continuous data that you really want for prediction. In fact, the biggest predictor of someone getting ill with a lot of wearable studies is missing data, because they were too sick to wear the sensor.

      Hanne: Oh, so the very absence of the data is a big indicator.

      Dr. Turakhia: Yes. Exactly.

      Hanne: That’s so interesting. I don’t feel well enough to put on my what-have-you, and that means something’s not right.

      Dr. Turakhia: Possibly, or you’re on vacation. And so that’s the problem. That’s other challenge of AI — is context. And so, what are some of the more simple problems where you have clean data structures, you have less noise, you have very clear training for these algorithms. And I think that’s where we’ve seen AI really pick up, in imaging-like studies. It’s a closed-loop diagnosis. You know, there is a nodule on an x-ray that is cancer-based on a biopsy, proven later in the training dataset, or there isn’t. In the case of an EKG, we already have expert systems that can give us a provisional diagnosis on an EKG. They’re not really learning. And so, that’s a great problem, because most arrhythmias don’t need context. You can look at it and make the diagnosis.

      Hanne: We don’t need them to learn, so that’s why it’s good to use right away, to apply this technology immediately.

      Dr. Turakhia: You don’t need everything. You don’t need to mine the EMR to get all this other stuff. You can look at the image and say, “Is it probably — does it have a diagnosis or does it not?” And so, imaging of the retina, imaging of skin lesions, x-rays, MRIs, echocardiograms, EKGs — that’s where we’re really seeing AI pick up.

      The importance of big data

      Brandon: I sort of divide the problems into inputs and outputs. We talked a little bit about some of the inputs that have become newly available, like EMR and imaging data. I think it’s also interesting to think about what the outputs of an AI algorithm would be. And these examples are self-contained, well-defined outputs that fit into the existing medical system. But I think it’s also interesting to imagine what could happen if you were to reinvent the entire medical system with this assumption that we have a lot of data — intelligence is artificial, and therefore cheap — so we can do continuous monitoring. So, one of the things I think about is, “What are the gaps of people who do not have access to EKGs?” I’ve actually never had an EKG done aside from the ones I do myself. So, and most people in the U.S. — you get your first EKG when you turn 65 as part of your Medicare checkup, and they won’t reimburse for anything after that.

      Hanne: Oh, wow, I didn’t realize it’s so late.

      Brandon: My dad’s an excavator, so he digs foundations for houses, and he hasn’t seen a doctor in 20 years. And if he leaves a job site, the entire job site would shut down. So, it’s hard for some people, I think, to go into the doctor’s office between the hours of 9:00 a.m. to 5:00 p.m. If you look at that in aggregate, about half of people in the U.S. have a primary care physician at all, which seems astonishingly low, but that’s the fact. There’s a gap — about a third of people with diabetes don’t realize they have it, about a fifth of people with hypertension, for AFib, it’s 30% or 40%. For sleep apnea, it’s like 80%.

      Vijay: I think it’s one thing just finding out but not being able to do anything about it, but the actionable aspect, I think, really is a huge game-changer. It means that you can have both better outcomes for patients, and, in principle, lower costs for payers.

      Hanne: Right. And these are areas where there are clear ways of addressing these specific conditions.

      Dr. Turakhia: I will take a little bit of a different view here, which is that I don’t know if AI — artificial intelligence — is needed for earlier and better detection and treatment. To me, that may be a data collection issue.

      Hanne: How is that different from what we’re saying about finding it early? How can that not be good?

      Dr. Turakhia: Because that may have to do with getting sensors out of hospitals and getting them to patients. And that’s not inherently an AI problem. It could be a last mile AI problem — so that if you want to scale the ability to get this stuff. So, let’s say we get to a point where our bathroom tiles have built-in EKG sensors and scales, and the data is just collected while we brush our teeth. It’s the sensing technology that may detect things discreetly, like an arrhythmia. You may not necessarily need intelligence, but who’s going to look at the data? And so that’s a scaling issue.

      Vijay: The AI could look at the data. And the other thing is, if you’re using this as screening, you want to make the accuracy as high as possible to avoid false positives. And AI would have a very natural role there too.

      Hanne: But it’s interesting that you’re saying it’s not necessarily about the analysis, it’s about where the data comes from and when.

      Dr. Turakhia: I think there are two different problems. There may be a point that it truly outperforms the cognitive abilities of physicians. And we have seen that with imaging so far, and some of the most promising aspects of the imaging studies and the EKG studies are that the confusion matrices — the way humans misclassify things — is recapitulated by the convolutional neural networks.

      Hanne: Can you actually break that down for a second? So, what are those confusion matrices?

      Dr. Turakhia: So, a confusion matrix is a way to graph the errors and which directions they go. For rhythms on an EKG, a rhythm that’s truly atrial fibrillation could get classified as normal sinus rhythm or atrial tachycardia, or supraventricular tachycardia. The names are not important. What’s important is that the algorithms are making the same type of mistakes that humans are doing. It’s not that it’s making a mistake that’s necessarily more lethal and just nonsensical, so to speak. It recapitulates humans. 

      To me, that’s the core thesis of AI in medicine. Because if you can show that you’re recapitulating human error, you’re not going to make it perfect. But that tells you that, in check and with control, you can allow this to scale safely since it’s liable to do what humans do. And so, now you’re automating tasks that, you know — I’m a cardiologist, I’m an electrophysiologist, but I don’t enjoy reading 400 ECGs when it’s my week to read them.

      Hanne: So, you’re saying it doesn’t have to be better — it just has to be making the same kinds of mistakes to feel that you can trust the decision-making.

      Dr. Turakhia: Right. You dip your toe in the water by having it be assistive, and then at some point, we as a society will decide if it can go fully auto. Fully autonomous without a doctor in the loop. That’s a societal issue. That’s not a technical hurdle at this point.

      Hanne: Right.

      Vijay: Well, you can imagine, just as — let’s say, self-driving cars, you have different levels of autonomy. It’s not nothing versus everything.

      Dr. Turakhia: It’s not.

      Vijay: You can imagine level one, level two, level four, level five in self-driving cars. I think that would be the most natural way because we wouldn’t want to go from nothing to everything.

      Dr. Turakhia: Exactly. And just like a self-driving car, we as a society have to define who’s taking that risk on. You can’t really sue a convolutional neural network, but you might be able to make a claim against the physician, the group, the hospital that implements it. You know, how does that shake out?

      Hanne: To figure out, literally, how to insure against these kinds of errors.

      Brandon: I think the way you think about some of these error cases depends on whether the AI is substituting for part of what a doctor does today, or if the AI is doing something that’s truly novel. I think, in the novel cases, you might not actually care whether it makes mistakes that would be similar to humans.

      Hanne: That’s an interesting point, because it’s doing something that we couldn’t achieve. What kinds of novel cases like that can you imagine?

      Brandon: Wearables are an interesting case, because they’ll generate about 2 trillion data points this year. So, there’s no cardiologist or team of cardiologists who could even possibly look at those. That’s a case where you can actually invert maybe the way the medical system works today. Rather than being reactive to symptoms, you can actually be proactive, and the AI can be essentially something that tells you to go to the doctor rather than something that the doctor uses when you’re already there.

      Vijay: Let’s take radiology as an example, where you could have one level where it’s as good as a common doctor, another level where it’s as good as the consensus of doctors.

      Hanne: Right.

      Vijay: Another level is that it’s not just using the labels [that a] radiologist would say on the image. It’s using a higher-level gold standard. It’s predicting what the biopsy would say. And so, now you’re doing something that…

      Hanne: Which would be into your kind of novel, plus…

      Vijay: Yeah, something that no human being could do. It can do that because in principle, it could fuse the data from the image, and maybe blood work, and other things that are easier to get and much less risk inducing than removing tissue in a biopsy.

      Hanne: So, pulling those multiple streams of information into one and sort of synthesizing them is another area that…

      Vijay: Yeah. It’s very difficult for a human being to do, and very natural for a computer.

      The unknowns of AI in healthcare

      Dr. Turakhia: It is very natural, but I think we need a couple of things to get there. We need really dense, high-quality data to train. And the more data you put in a model — I mean, so, machine learning, by definition, is statistical overfitting, and sometimes…

      Vijay: Well, actually, I think that’s wrong. Machine learning done poorly — I mean, it’s like saying driving is driving a car off a cliff. Poor driving is poor driving, but machine learning tries to avoid statistical overfitting.

      Dr. Turakhia: It does. My point is that one of the unknowns with any model, it doesn’t matter if it’s machine learning or regression or a risk score, is calibration. And as you start including fuzzy and noisy data elements in there — first of all, often the validation data sets don’t perform as well as the training dataset, no matter what math you use.

      Hanne: Okay. And why is that?

      Vijay: Well, that’s a sign of overfitting, and usually it’s because there wasn’t sufficient regularization during the training process.

      Dr. Turakhia: So overfitting is a concept in statistics to effectively indicate that your model has been so highly tuned and specified to the data you see in front of it, that it may not apply to other cases.

      Hanne: It can’t generalize.

      Dr. Turakhia: If you had to use a model to identify a bunch of kids in a classroom and pick the kid who’s the fastest, an overfitted model might say it’s the red-headed kid wearing Nikes. Because in that class, that was the case.

      Hanne: That was the one child.

      Dr. Turakhia: But that has no plausible biological or other plausibility.

      Hanne: You can’t extrapolate, it doesn’t generalize, yeah.

      Dr. Turakhia: You can’t use that. If you take that to a place where the prevalence of Nike shoes or redheads is low, you might miss the fastest person, right?

      Hanne: Not helpful, yeah.

      Dr. Turakhia: These are some of the issues. The underlying shifts in population, the natural language processing that’s embedded in AI, the lexicon that people use. How doctors and clinicians write what it is that they’re seeing with their patient is different, from not even specialty to specialty, but hospital to hospital, sort of mini subcultures.

      Brandon: It’s going to be different at Stanford than it was at UCSF, which is going to be different at Cleveland Clinic. I think that’s actually a nice thing about wearable data, is that Fitbits are the same all over the world. This label problem though is interesting because, in our context, each label represents a human life at risk. It’s a person who came into the hospital with an arrhythmia, and so you’re not going to get a million labels the way you might for a computer vision application. It would be unconscionable to ask for a million labels in this case. So, I think one of the interesting challenges is training deep learning-based models, which tend to be very data-hungry, with limited label data.

      Dr. Turakhia: The kitchen-sink approach of taking every single data element — even if you’re looking at an image, can lead to these problems of overfitting. And what Brandon and Vijay are both alluding to is, you limit the labels to really high-quality labeling, and see if you can go there. And so don’t complicate your models unnecessarily.

      Vijay: And don’t build models that are overly complicated for the amount of data that you have. Because if you have the case where you’re doing so much better on the training set than the test set, that’s proof that you’re overfitting, and you’re doing the ML wrong.

      Brandon: Modern ML practitioners have a whole set of techniques to deal with overfitting. So, I think that problem is very solvable with well-trained practitioners. One thing you’ve alluded to, which is the interpretability aspect. So, let’s say you train on a population that’s very high in diabetes, but then you’re testing on a different population, which has a higher or lower prevalence. That is kind of interesting — so, identifying shifts in the underlying data and how you get…

      Hanne: What would that mean?

      Brandon: So, let’s say we train on people in San Francisco, and everyone runs to work and eats quinoa all day. But then we go to a different part of the country where maybe obesity is higher, or you could be somewhere in the stroke belt where the rate of stroke is higher. It may be that the statistics you trained on don’t match the statistics that you’re now testing on. It’s fundamentally a data quality problem. If you collect data from all over the world, you can address this. But it’s something you have to be careful with.

      Hanne: But it will take a while for that to happen as we start gathering the data in different ways. How does that actually even happen? How are these streams of data funneled in and examined and fed into a useful system?

      Brandon: So, used to be, the way you’d run a clinical trial, you would have one hospital. You’d recruit patients from that hospital, that’d be it. If you got a couple of hundred patients, that might actually be quite difficult to attain. I think with ResearchKit, HealthKit, Google Fit, all of these things, you can now get 40,000 or 50,000 people into a study from all over the world, which is great, except the challenge that the first five ResearchKit apps had is that they got 40,000 people, and then they lost 90% of them in the first 90 days.

      Hanne: So, everybody just drops out?

      Brandon: Everyone just drops out, because the initial versions of the apps weren’t engaging. So, this adds an interesting new dimension. As a medical researcher, you might not think about building an engaging, well-designed app, but actually, you have to bring mobile design in as now a discipline that you’re good at.

      Hanne: So, there has to be some incentive to continue to engage.

      Brandon: Yeah, exactly. You need to measure cohorts the same way Instagram, or Facebook, or Snapchat does. So, I think the teams that are starting to succeed here tend to be very interdisciplinary. They bring in the clinical aspect, because you need that to choose the right problem to solve, but also design the study well so that you have convincing evidence. You need the AI aspect, but you also often need mobile design, if it’s a mobile study. You may need domain expertise in other areas if your data is coming from somewhere else.

      Hanne: And then it all has to be gamified and fun to do.

      Brandon: Yeah. Well, I mean, gamification is sort of extrinsic motivation, but you can also give people intrinsic motivation — giving them insights into their own health, for instance. It’s a pretty good way to hook people.

      Urging the current system to adopt AI

      Hanne: What’s the system’s incentives? I mean, of course, doctors want it if it makes them more accurate or to scale better. Patients want it if you can predict whether or not you’re going to have a problem. How do we incentivize the system as a whole?

      Dr. Turakhia: I believe fundamentally it is going to come down to cost and scale, and what willingness does a healthcare entity, whoever that may be — whether it’s employer-based programs, insurer-based programs, accountable care organizations. Are they going to be willing to take on risk to see the rewards of cost and scale? And so, the early adopters will be ones who’ve taken on a little more risk.

      Vijay: Yeah. I think, you know, it is — the challenge is where the hope is, and in terms of value, and in terms of better outcomes. But one has to prove it out and hospitals will want to see.

      Dr. Turakhia: The regulatory risk thing is being largely addressed by this new office of digital health and the FDA, and they really seem much more forward-thinking about it. But there are going to be challenges that we have to solve, and I’ll give you one just to get the group’s input here. Should you be versioning AI, or do you just let it learn on the fly? And so, normally, when we have firmware, hardware, software updates in regulated and FDA-approved products, they’re static. They don’t learn on the fly. If you keep them static, you’re losing the benefit of learning as you go. On the other hand, bad data could heavily bias the system and cause harm, right? So, if you start learning from bad inputs that come into the system for whatever reason, you could intentionally or unintentionally cause harm. And so, how do we deal with versioning in deep learning?

      Vijay: I mean, to just freeze the parameterization — so versioning, from a computer science point of view, is trivial. There’s the deeper statistical question, which you could version every day, every week, every month.

      Hanne: Right. It’s when and how often.

      Vijay: And just freeze the parameters. What you want to do is, to the point we were talking about earlier — you want to bring in new validation sets. Things that it has never seen before, because you don’t want to just test each version on the same validation set, because now you’re intrinsically overfitting into it. What you always want to be doing [is] holding out bits of data [so that] you can test each version separately, because I want to make sure that they have very strict confidence that this is doing no harm, and this is helpful.

      Hanne: Right. It’s like, we’re introducing this whole new data set of a different kind of thing, and that’s when you make new considerations.

      Vijay: Yeah. Data’s coming in all the time, and so you just version on what came in today, and that’s it. It’s pretty straightforward. And as you’re training it…

      Brandon: This is the way speech recognition works on Android phones. Obviously, data is coming in continuously, and every time someone says, “Okay, Google,” or, “Hey, Siri,” it’s coming into either Google or Apple. But you train a model in batch and then you test it very carefully and then you deploy it. The versions are indeed tagged by the date of the training data.

      Hanne: It’s already embedded in the system. Who are the decision-makers that are green lighting when, like, “Okay, we’re going to try this new algorithm. We’re going to start applying this to these radiology images.” What are the decision points?

      Dr. Turakhia: So, with EKGs, the early companies used expert systems to just ease the pain points of me having to write out and code out every single diagnosis.

      Hanne: The super low-hanging fruit.

      Dr. Turakhia: Yeah. Can you improve the accuracy of physicians with this? Can you increase their volume and bandwidth? Can you actually use it to see which physicians are maybe going off course? What if you start having a couple of physicians whose error rates are going up? 

      Right now with quality, the QI process isn’t really based on random sampling. There’s actually no standardized metrics for QI in any of this. When people read EKGs and sign them off, they just sign them. There’s nothing telling anyone that this guy has a high error rate. And so, that is a great use case of this, where you’re not making diagnoses, but you’re helping anchor and show that, well, if you believe this algorithm is good and broadly generalizable across data, you’re restating the calibration problem now.

      It’s not that the algorithm has gotten necessarily worse, because, in fact, in seven of the eight doctors, it’s right on par with them. But in this other doctor, it could be if that doctor — if that doctor is not agreeing with the algorithm which is agreeing with the other seven, that doctor is actually not agreeing with the other seven. So now you have an opportunity to train and relearn. Those are the use cases to go off of.

      Vijay: You can train and relearn the person?

      Dr. Turakhia: The person. Address their reading errors, coding errors, see what’s going on. And that qualitative look, I think, is very, very valuable.

      Hanne: So, what are the ways we’re actually going to start seeing it in the clinical setting? You know, the tools that we might see our doctor actually use with us or not.

      Dr. Turakhia: I think it’s going to be these adjacencies around treatment with management. There are a lot of things that happened in the hospital that seem really primitive and arcane, and no one wants to do them. I’ll give you a simple one, which is OR scheduling.

      Hanne: So, is it actually the way it looks like it is in “Grey’s Anatomy?” Is it just a whiteboard and an eraser?

      Dr. Turakhia: It is a whiteboard and somebody on the phone. The OR front desk.

      Hanne: That’s unbelievable.

      Dr. Turakhia: There’s a backend of scheduling that happens for the outpatients, but you have add-ons, you have emergencies, you have finite…

      Hanne: I mean, it seems like even an Excel sheet would be better than a whiteboard.

      Dr. Turakhia: The way OR scheduling works now is primitive, and it also involves gaming. It involves convincing staff X, Y, and Z to stay to get the case done, or do it tomorrow.

      Hanne: So, there’s so much behind the scenes, like, human negotiation?

      Dr. Turakhia: When I do catheter ablations, we have many different moving parts. Equipment, the support personnel of the equipment manufacturer, anesthesia, fellows, nurses, whatever. Everyone has little pieces of that scheduling. It all comes together, but it comes together in the art of human negotiation and very simple things like, “This is your block time, and if you want to go outside your block time, you need to write a Hallmark card to person X.” So, very simple problem where there’s huge returns inefficiency if you could have AI do that. With the AI inputs over time, you could be like, well — you can really know which physicians are quick and speedy, which ones tend to go over their allotted times, which patient cases might be high risk, which ones may need more backup, which should be done during daytime hours.

      Brandon: You could add their Fitbit data and then you could tell who’s drowsy at any given moment, for a little elaboration there.

      Hanne: Oh, that’s fascinating. Yeah. Whether or not they want to do it.

      Brandon: How stressed are they feeling.

      Dr. Turakhia: And so, people stay at times that they’re really needed. That kind of elasticity can come with automation where we fail right now. This is a great place where you are not making diagnoses. There’s nothing you’re being committed to from a, kind of, basic regulatory framework. You’re just optimizing scheduling.

      Hanne: So, who actually — so, say that that technology is available. How do you actually get it in — where’s the confluence of the regulation and the actual rollout, and how does it actually make its way into a hospital and to a waiting room?

      Brandon: There’s an alternative model I’ve seen, which is startups acting as full-stack healthcare providers. So, Omada Health or Virta Health would be examples of this, where if you have pre-diabetes or diabetes, respectively, the physician can actually refer the patient to one of these services. They have on-staff physicians. They’re registered as providers with national provider IDs. They bill insurance just like a doctor would, and they’re essentially acting as a provider who addresses the whole condition end to end. 

      I think that case actually simplifies decision-making, because you don’t necessarily have to convince both Stanford and United Healthcare to adopt this thing. You can actually convince just a self-insured employer that they want to include one of these startups as part of their health plan. And so, I think that simplifies the decision-making process and ensures that the physicians and the AI folks are under the same roof. I think that’s a model that we’re going to see probably get the quickest adoption, at least in the <inaudible> world.

      Vijay: There are many models, and which is the best model will depend on how you’re helping in the indication, and on the accuracy, and what you’re competing against, and so on.

      Brandon: This is a case where, probably, we’ll see the healthcare industry maybe reconstitute itself by vertical, with AI-based diagnostics or therapeutics. Because, if you think — right now, providers are geographically structured. But with AI, every data point makes the system more accurate. Presumably, in an AI-based world, providers will be more oriented around a particular vertical. So, you might have the best data network in radiology, the best data network in pathology, the best data network in…

      Hanne: That’s interesting. Yeah. Thank you so much for joining us on the “a16z Podcast.”

      Vijay: Great. Thank you.

      Dr. Turakhia: Thank you.

      Brandon: Thanks for having us.

      • Brandon Ballinger

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      • Mintu Turakhia

      • Hanne Winarsky

      Revenge of the Algorithms Over Data

      Sonal Chokshi, Frank Chen, and Steven Sinofsky

      There are many reasons why we’re in an “AI spring” after multiple “AI winters” — but how then do we tease apart what’s real vs. what’s hype when it comes to the (legitimate!) excitement about artificial intelligence and machine learning? Especially when it comes to the latest results of computers beating games, which not only captures our imaginations but has always played a critical role in advancing machine intelligence (whether it’s AI winning Texas Hold’em poker or beating the world human champ in the ancient Chinese game of Go).

      But on learning that Google DeepMind’s AlphaGo can master the game of Go without human knowledge — or more precisely: “based solely on reinforcement learning, without human data, guidance, or domain knowledge beyond game rules” — some people leap too far towards claims of artificial generalized intelligence. So where can we then generalize the findings of such work — unsupervised learning, self-play, etc. — to other specific domains? What does it mean for entrepreneurs building companies (and what investors look for)? And what does it mean for how we, as humans, learn… or rather, how computers can also learn from how we learn?

      Deal and research operating team head Frank Chen and a16z board partner Steven Sinofsky ponder all this and more, in conversation with Sonal Chokshi, in this episode of the a16z Podcast. We ended last time with the triumph of data over algorithms and begin this time with the triumph of algorithms over data … is this the end of big data?

      Show Notes

      • Discussion of how AlphaGo’s algorithm learned Go [0:45], and the potential limits of machine learning [9:41]
      • Questions of humans interacting with AI, and our expectation of accuracy [18:46]
      • Potential bias in algorithms [29:46] and final thoughts about the future of AI/ML [34:21]

      Transcript

      Sonal: Hi, everyone, welcome to the “a16z Podcast.” I’m Sonal. And I’m here today, bringing back the band together again — the AlphaGo band, I guess — I don’t know how else to describe us. But we have the head of our Deal and Research Team Operation, Frank Chen. We have Steven Sinofsky, a16z board partner.

      Just to give people some quick context, you don’t have to have heard our previous podcast on AlphaGo. But when AlphaGo, the algorithm produced by DeepMind, beat the world champ in Korea in playing Go, which is an ancient Chinese game, we had a podcast where we discussed a lot of the themes and some of the broad things around that. And what we’d like to talk about today is their latest paper — but not just only specifically, but more broadly — what this means for where we actually are. What’s hype, what’s real in AI or artificial intelligence. So, welcome, guys.

      Background on the AlphaGo algorithm

      Steven: Welcome. So, like one thing that, like — you read this paper, and the paper is published in “Nature.” It’s pretty dense. It has 17 authors on it. And so, it’s quite the force. But the thing that sort of jumps out is the paper, the blogs, everybody — is, the first thing you read is, “one step closer to creating general purpose AI.” And immediately, like, my AI winter, fears of hype antennae pop up, because, like, everything is one step closer. But like, you could take very, very, very tiny little steps, or you could overhype them.

      Frank: And we’ve heard these promises throughout history, and especially around board games. So, we solved checkers, and people were like, “Oh, one step away from general intelligence.” Then we solved chess, same thing, we’re one step away from general intelligence.” Then we did the, “Oh, I can find bacteria that causes infection; therefore, we must be one step away.”

      And I think the fallacy is that because we’re doing these things that are considered, sort of, high cognition — like the smart people play chess, smart people figure out…

      Sonal: Yeah, strategy.

      Frank: …then solving one thing that a smart person does must lead us to the next thing that a smart person does, which will lead us to the next. And that’s always been the fallacy, which is, it actually hasn’t quite generalized.

      Steven: Well, in fact, they don’t even really compound all that much. They’re all fairly discrete. The one thing that’s different now is that they are all building on these new artificial intelligence or machine learning techniques, and taking a whole bunch of data, training a model on it, and developing solutions that beat all their algorithmic ones. And that was a big thing about the first AlphaGo was, like, “This is data over algorithms.” And all of a sudden, here we are…

      Sonal: Algorithms over data.

      Steven: Or, a model over data. But you know, Frank, you have an interesting way of looking at this, because, you know, you play Go, and you understand that…

      Sonal: Oh, you play Go? I didn’t know that.

      Frank: Yeah. Well, enough, yeah.

      Steven: More than me.

      Sonal: I only play chess. I don’t even know…

      Frank: Yes, Chinese chess and Go. All right, I play a few games.

      Steven: But, like, it’s not like a generalized problem. You know, it comes with, like, a whole bunch of constraints and things that make it solvable by algorithm.

      Frank: Yeah. So, what are some traits about Go that are not like the real world? All the rules are completely well known. The state of play is completely well known, right? In the real world, mostly, we live in a “fog of war” situation, where we know some things, and we don’t know other things. It would cost us something to go figure it out. In Go, and in particular the representations they chose, the entire board state is known to both players, right? So, there’s a lot of things that aren’t like the real world at all when we do these board games. Now, having said that, why are people getting breathless, yet again, around one step closer that AI…

      Sonal: I think, because the authors don’t claim generalized intelligence. They’re talking about being able to do — apply some of these techniques in other domains.

      Frank: Yeah, so to give credit where credit’s due. Let’s, sort of, review what they actually achieved. And they are, like, very impressive achievements. So, what they achieved was, they now have a Go player that beat all of their other Go players.

      Sonal: AlphaGo Zero.

      Frank: Well, they named them by the people they beat. So, they had AlphaGo Fan, AlphaGo Lee. This one’s AlphaGo Zero. So AlphaGo Zero, their latest one, has beat every iteration, which has beat all of the best human players, right? So, they have the best Go-playing algorithms in the world. And the difference in how they got to this one is, there was no training data from human games. So, all the other ones had been bootstrapped by, “Let me go watch what the human players do and see if I can mimic that.” And that, very broadly speaking, is the approach to machine learning called supervised learning.

      Steven: And not a trivial amount. I mean, they had 100,000 games that they started from to train it on, like, 48 of Google’s TPUs for days on end.

      Frank: That’s exactly right. So, a fleet of machines, a ton of data. This one started from nothing, which is, it didn’t even really know the rules. It had a loss function, which is reinforcement learning’s way of sort of improving the algorithms over time. So, let me just give a quick intuitive explanation of reinforcement learning. So, it’s exactly like the game that you used to play when you were a kid and called hotter or colder. Somebody would go hide something, and then you would try to get to discover it.

      Sonal: Are you getting hotter? I’m getting hotter, hotter, cold, cold again.

      Frank: Exactly. And then, the hotter or colder is the loss function. That is the thing that tells you if you’re getting closer or not.

      Sonal: It’s basically trial and error, simply put.

      Frank: That’s exactly right. And so, that’s the fundamental approach in AlphaGo Zero, which is, they have this loss function that describes, are you more likely to win the game having made this move or not, right? Ao, that’s all they had. And then they didn’t have human input. They didn’t have human games. They basically said, “I have this loss function that tells you whether you’re more likely to win this game or not.” And then it played itself.

      Steven: One other important thing, I think, it has — it also knows — it has the codified rules of the game.

      Sonal: That is the one human input it actually got.

      Steven: Well, right. And it makes it a very, very constrained problem. Because there’s a whole bunch of the decision trees — like, hot or cold, you could climb up the sofa, you could leave the house, you could go all over the place for days on end. Whereas, this just tells, “No, it’s gonna only be on the floor.”

      Sonal: These are the rules of the game.

      Steven: Like, this is not gonna be under the sofa.

      Frank: It’s a 17 by 17…

      Sonal: Very little of real life is actually, like, you get the real Shogi that way.

      Frank: Yeah, it’s a 17 by 17 board, it’s black and white pieces, there are rules.

      Steven: Oh, yeah, you switch turns, like all of that stuff matters a lot. 

      Sonal: It does. And one more thing to add, by the way, too, because you mentioned the 48 TPUs — it’s really significant that they got it down to 4 in this case. That’s a huge, like — on the power side energy, like — you know, it’s a simpler architecture.

      Frank: It’s a simple architecture, and thinking from the point of view of a software developer, now you’re back to one machine. You’re not like, “Oh, my God, I need to go rent this massive cloud with massive storage and massive interconnects. And, like, I need to figure out how to provision the cluster and manage the cluster.” You’re back down to one machine, right?

      Steven: So, this stuff was pretty impressive. It’s, you know, they did four TPUs. It was three days of playing.

      Sonal: Three days total.

      Steven: Like, it’s very achievable, you know, like, on your Amazon credits.

      Frank: That’s exactly right. And if you think about sort of the approach that most startups take to artificial intelligence today, they basically take the supervised learning approach. And step one, raise money so that you can go get a data set that’s annotated, train your neural network, make recommendations, right? And you could be $1, or $5 or $10, or $50 million in getting that data set, depending on how complicated the data set is. All right, so the reason people are so excited about this is, look, this had no data, aside from rules of the game. It basically played itself. And by day three, it was better than everything that had been trained before it.

      Sonal: Nearly an order of magnitude.

      Frank: Yeah, it was, orders of magnitude on this…

      Sonal: And the results would be a hundred to zero.

      Frank: …48 TPUs versus 4. It was 3 days versus 40. It was 30 million train games versus 4.9. So, order of magnitude improvement on all of those dimensions, so, like, let’s give credit where credit is due — this is a very impressive technical achievement. And then, the question that we sort of entered the session with, “Okay, does this make us more likely to be able to create an artificial general intelligence, where the learning algorithm is generalized across domains?” In other words, can I take the breakthroughs here, and make a better pick and pack robot for Amazon? Or make a better healthcare predictor to discover whether you have cancer?

      Steven: Or Salesforce forecasting, code generation, or a whole bunch of stuff. And the interesting thing is that — also the techniques here — this is just proof of something that people have been talking about for a long time.

      Sonal: Oh, yeah, reinforcement learning has been around for ages.

      Steven: And so, part of what’s interesting to me is that reinforcement learning has been around for a long time. Obviously, the modeling has been around for a long time…

      Sonal: Self-play.

      Steven: …self-play, all of these things. And, like, so many times these steps are, like, somebody new to the domain looking and pulling together a bunch of unrelated things, and just coming up with a very elegant, incredibly elegant solution. And it’s super impressive, but it’s not clear it generalizes. And I think that was one of the things that jumped out for me in reading about it, you know. When you hear, “Oh, and then the next step, on this train of generalized intelligence is drug discovery, and protein folding, and quantum chemistry, and material…” And it’s like, all of a sudden, I’m trying to figure out — protein folding. Like, what are the constraints on protein folding? Well, we know they’re amino acids, and we know that they have to be in three dimensions. But actually, nobody else knows…

      Sonal: There’s no codified rules. 

      Steven: Like, nobody has the rules of protein folding.

      Frank: So, there’s no rules, there’s no perfect understanding of the search space. There’s like — what’s the loss function? Like, how would you even write a loss function?

      Sonal: I do want to push back a little bit, though, because far be it from any of us in here to hype this up. But there’s something unique happening here, which at least — I perceive this in the paper. Which is, that I was struck by the analogy to evolution. Like, this is how human beings have evolved. This is evolution — that we learn by trial and error on a massive million scale. So, I don’t want to completely dismiss the idea that we can get to some kind of generalized intelligence. I mean, of course, I understand and agree, but what are the limits? And what are the possibilities that can actually take us there? And where are we constrained? Just to break that down a bit more.

      The limitations of machine learning

      Frank: Yeah. So, I love the evolution analogy, right? Because in the paper, they talked about how, you know, when it started out, it was making, sort of, naïve moves. It was, sort of, greedy and trying to capture all the tokens. And then, it got to very sophisticated patterns that humans have discovered over thousands of years playing this game, teaching this game, codifying it in books. And it figured that out. And not only did it figure it out, it figured out things that humans haven’t quite codified, right?

      Sonal: Right. New ways of playing.

      Frank: If you play more games, sort of, it developed an intelligence that humans haven’t yet, because it’s going through its own reinforcement thing. And thousands of generations of games playing each other, sort of, arrive at places that maybe humans would have gotten to if we played another thousand years. But like, you know, it figured out in three days. So, I think there is something incredibly profound going on here, where you’re, basically — you’re accelerating natural selection cycles in computers.

      Now, where I think the analogy breaks down is, “Oh, and therefore we can apply this approach to every other problem.” And it’s just going to be a straightforward application of the set of ideas to those problem sets. And then, we’re going to have evolution at that scale.” In other words, eating different problems, right, in exactly the same way. And I think that’s where Steve and I have a little skepticism.

      Steven: Yeah, well, and I think also like the key with all of these, if you just take an abstract view of it, you know, you have this awesomely elegant solution to an intractable problem, which is just, on any measure, super cool. But that doesn’t make it generalizable to any other space. There are many, many super cool things that surface. And you have to be careful as, like, an engineer, or a founder, or a person applying this to [go], “Okay, well, what are the elements of this solution that one would need to have as a precursor to applying it.”

      And we saw the same thing with all of the work on supervised learning. Like, first you need a data set that is clean, and then it has to be, like, labeled really well. And then, you have to have a neural network model. And you have to do all the weights and all this modeling. And so, there are all these things where you couldn’t just say, “Hey, I’d like to make the most money in our Q4 sales. Let’s machine learn our way there.” And you, like, you can’t just show up on Thursday and do that. Every year, salespeople — they develop a model for how they want to sell, how the customers — what the prospects are. They know the rules — like, there are all these rules. Like, there are this many salespeople, they can only call so many people per day. They know which customers to call. They know what the quotas are going to look like. They know what the product — and you start to think, well, maybe there is something here, because it’s a space where, like, actually, the history might not be as well as applicable to supervised learning as you would like. And so, if there was a way to look at this through the lens of, like, what would be the optimal system to navigate? Or what would be the sales matter? And I think that there are more things like that than fewer. I don’t know how many, you know, properties of nature are amenable to this. Because, for the most part, we don’t understand them.

      Sonal: Right. It’s limited by what the human knows, right? And we can’t codify those rules. What are the domains that you see some transferability of these types of things?

      Frank: Well, I love the idea of, sort of, sales forecasting, right? Because, essentially, what you’re doing — the intuition is that if I could play act my salespeople doing a million different things, in a million different situations, against a million different set of prospects, you know, I could sort of simulate that, then maybe best practices would just come out of that, right? Because, essentially, that’s what I’m doing in real life. I raise a ton of money, I hire a sales leader, that sales leader hires a bunch of people, gives them a playbook, and has them call a bunch of prospects. I can only run so many experiments, right? Because every call is an experiment. So, the idea would be, if I could simulate what happens inside these calls, and simulate 10, 100, 1000, a million times more, then I’d get much better best practices emerging from that. So, super intriguing idea.

      Steven: Off the top of my head, you start wanting to think about, like, “Okay, what about, you know, cybersecurity.” And you think, you’re looking at your code, and you know at any given moment, like, the type of code that can be in a specific place. And you know, the rules and the syntax, and it’s well understood, and you know patterns that are also bad. And so, today, what you do — and the people that have tried to apply machine learning to code — they just have a lot of examples of, like, missing equal signs, or missing semicolons, or operators are wrong. But if you think about it, like, the syntax of the language is completely fixed. And so, you’re back at very much a…

      Sonal: A rule based…

      Steven: A rule based kind of — you know, rule base is a constraint of, like…

      Sonal: Constraint, right.

      Steven: These are the only ways you can put the…

      Sonal: Definable constraint.

      Steven: …symbols together. What you don’t have is right or wrong, win or lose with code.

      Frank: Yeah, you don’t have a loss function that’s that objective.

      Steven: They’re interesting things to me that, like, there’s just complexity of line that is a very common measure. Like, “Boy, something with three sets of parentheses in it, it’s likely to have a bug in it just because it has three sets of parentheses in it.” You know, something that’s missing, you know, some enclosure kind of rules, like using brackets, not using brackets. These kinds of things, you can actually flag, like — you can think of lint. Just flags them in C and C++ as just, sort of, high-risk behaviors.

      So, there are things that you can put in to, like, sort of — and so then you start to think, “Wow, it’d be really interesting to have a tool that is able to look at code, and, sort of — just very different than previously, which was just literally looking at the syntax — but doing millions of examples of generating code and finding bad examples.” Would it be able to do a better job at finding bad examples in the next piece of code that falls into it?

      Sonal: Right. I mean, one of the points you guys made last time, in our last podcast is that, at the end of the day, these things aren’t working in isolation. It’s not like there’s one magic approach. You know, there’s always a combination of techniques that come together to actually build real products. How would this, sort of, fit into that? Because one of the thoughts that I had is that — clearly, this kind of approach, even if you don’t have clearly defined rules, will always be more beneficial in places where we don’t have any data. Like, any big data. Just like humans, like kids, learning from N equals 2, like their parents. Or N equals 1, if they’re a single parent.

      Frank: I think if we think about what humans do, they have many different types of intelligences. They have many different types of strategies for solving problems. So, my guess is that the artificial intelligences that we create will be similar. Lots of different strategies. And one of the interesting research items is, which strategy should I employ to solve this problem? Because this is something the brain does effortlessly. The strategies it employs to understand conversation are different than planning a trip, than, you know, making sure you don’t fall down, versus, like, long-range planning. Like, how do I choose the best career, right? So, all of these are very different problems. And somehow your brain kind of picks a good strategy for each one. Pedro Domingos talks about this as, sort of, you know, in his book, “The Master Algorithm,” which is — we know that there’s all of these different techniques, but kind of what we’re missing is sort of the synthesizer. The thing that we’ll know…

      Sonal: Yes, putting it all together.

      Frank: …what strategy should I pick to solve this problem? And it’s something that the brain seems to just do.

      Steven: What’s interesting about that observation is, that’s where we are with machine learning today to begin with.

      Sonal: Which is?

      Steven: Which is just, like, there are all of these different networks that you can model, with so many different layers and how many parameters you want to use. And there’s, like, this art to it right now. And I find that particularly interesting, because anytime there’s an art to something, there’s an opportunity to start a company around that art and to build out a product that surpasses, you know, the best practices in whatever field you’re going after, whether it’s analyzing traffic or figuring out how to drive a car. And so, that is the opportunity, because I don’t think, you know — there’s not, like, some path where in the next two years, there’s the meta-algorithm that knows what way to pick, like, that’s — so, the best thing to do now is to become well versed in all of them.

      And my gut just tells me that anytime you’re at a point like this, the most interesting solutions are actually going to end up being a hybrid of, like, the thing that used to work, that everybody said doesn’t work well enough, plus the new thing that everybody says is gonna replace the old thing. And that’s basically been the entire history of AI.

      Sonal: Computing in general.

      Steven: Well, in computing, for sure, but AI specifically, which is, like, everybody always says, “The new thing — that’s it, we finally got it. Huge step, the old stuff is all done. And the best example for me of that is how everybody said machine learning was going to replace all of natural language processing. But if you dig into any of the work that’s been going on, even the most state-of-the-art translation, which, you know, goes any language pair to any language pair — well, the input and the output all rely on the old school, like, from the 1970s, natural language stuff, just to do some very basic bookkeeping, very basic stuff.

      Sonal: Right. You talk about, like, building a spell checker, whatever it is.

      Steven: I mean, all the…

      Frank: Parts of speech finder.

      Steven: Right, and all the image stuff. Like, wow, you know, if you want to figure out features, it turns out, when you start doing features of image recognition, you’re using a whole bunch of old school edge detection and contrast and finding objects and all the stuff. And so, you can’t just, like, show up and say, “Now, we’re going to understand — we’re going to be the unsupervised learning company.” Because the question I was gonna ask is, well, how are you going to make whatever you’re doing practical?

      Human interactions with AI/ML

      Sonal: What I do remember about hearing the stories of the early days of NLP, and observing parts of this firsthand as well, is how the entire field and community — a lot of them had very strong opinions about, you know, there’s this whole phase of like, expert domain knowledge building. And really, that’s the only way to actually make NLP work at scale. There were all these things they had to do because it was before the days of big data. They couldn’t even conceive of the Google scale big data. And then, they went to this world where, “Oh, my God, we don’t even have to have these kinds of constraints and ways of doing things, because we have all this data.” And now, it’s sexy to me that you can flip that model again and almost say, “You don’t even need big data with the results like this kind of paper because you don’t have to have anything.”

      Frank: This is the revenge of the algorithm’s movement, which is, it’s always been about the data. If we had more data, we’d have more accurate models. And what this experiment showed is, “Look, all I needed was the rules and a really good loss function, some very clever programming, and I get better performance than I had when I had the data set.” Right? So, that’s the tantalizing, “Oh, look, you could do it without data.” And like, look, as an investor, that’d be awesome. If I could fund AI experiments without this step of — collect data, annotate data, train model — we could run a ton more experiments

      Sonal: Exactly. Because it’s age of abundance. You brought up Pedro Domingos, and it was funny because one of the comments he made on the heels of this announcement, which I thought was quite interesting, is, like, “Well, you know, AlphaGo Zero learned after, like, 5 million games, humans took only so many thousands of years.”

      Steven: I love that.

      Sonal: And I’m sort of like, “That’s the whole point of computers, is to learn it in three days.” You know, time is money. I don’t care if it took, like, you know, 5 million games, it took three days. But time is more important than amount.

      Steven: You know, again, it’s an amazing accomplishment. It’s just, I always try to look at these in the context of how these innovations tend to happen. You know, if you just look, the same thing happened with search, like search itself. When people were inventing all the techniques of search, they were working on these tiny machines with all of these physics constraints about how much they could compute. And like, it was a whole thesis, just to be able to go get, like, the archives of “The New York Times” to search through it.

      And the same thing happened with spell checkers. Like, wow, we’d love to have 50,000 words in the spell checker, but we don’t have that much memory. And then, there was a whole — well, now we have a lot of memory, and so now we’re spending all this money to try to compile all the words. And then, someone said, “Why don’t we just use all of the internet as the spelling dictionary. Then there’s no spelling dictionary, or the IME if it’s an Asian language. And so, this curve just keeps repeating itself. And I think that it — neither is going to win out, because data is always going to be valuable, because it’s an input that you can’t just say…

      Sonal: Yeah, it’s a great way of thinking about it.

      Steven: …whereas, the flipside is, not everything has data. So, if there’s this opportunity — like, you know, a great example is, like, how many of your Lyft drivers also have Waze turned on, just because they know the maps are entirely accurate, but they don’t know about accidents, or emergency closures, or weather, or whatever. And so having that data, combined with the data plus algorithms, that hybrid…

      Sonal: Is always gonna win, right.

      Steven: …is gonna continue to win, because only if you really want to be practical and actually solve the problem.

      Sonal: I have to say one thing about this, which — it’s so fascinating that you use the example of search, and you describe this curve that we’re on. And part of startups is, sort of, you know, getting at the right place and point in time along that curve and where you are in the moment. Because sometimes, I think a lot of academics, or even people who are really in love with their own ideas, sometimes lose a big picture. Like they’ve had this build up, this expertise in one area. And they don’t realize, like, practically speaking, the world has changed around you. Because the point that I think is fascinating, as well, in the innovation story, using the search example, is that Google was, like, the 15th search company to come around before it hit success. And that is kind of relevant to think about.

      Steven: Right. And it’s super important too, from the company building perspective, which is — they have this algorithm, which we all talked about back then. PageRank, you know, appropriately named but not…

      Frank: After Larry.

      Sonal: Or web page ranking as well.

      Steven: Right, right. But the interesting thing was very rarely is, like, an algorithm, like, this secret sauce for a company. Because you could look at what goes on from the outside and pretty much reverse engineer an algorithm. So, again, back to — but if you have, like, a data source that you can actually…

      Sonal: And a monetization model.

      Steven: And turns out, one of the things that Google built out — was, like — they were crawling the web faster than AltaVista. They made this bet because they were machine learning people before machine learning was cool. They made the bet that having the data was going to be — and, it turns out, that was the barrier to entering the search market. Even for Microsoft and Bing [it] was, like, sucking in the entire internet fast enough.

      Frank: Step one, crawl the inner web.

      Steven: Step one, crawl the internet while it’s growing at exponential rates. And so, you know, I actually want to bring up one more thing that I just think is, kind of, really interesting from a practical point of view.

      Sonal: I love it, you’re like the practical person on this podcast.

      Steven: I know. I feel, like, particularly practical today because, well, I can’t play Go, so I gotta…

      Sonal: You got to add something. I’m just kidding.

      Steven: Which is, the thing that I find the most fascinating about all of these solutions in the space, is the engineering of a product — that you can make a commitment to customers that works. And then, when it doesn’t work, you can figure out why. And so, one of the things that’s so interesting about all of this is debugging.

      Sonal: Hmm, interesting.

      Steven: And how does that really fit in? And sure, you know, with Go, you lost the game. And of course, while they’re building all of this, they’re figuring out, “Whoa, what did we do wrong to make this move repeatedly?” They’re doing all of that debugging over the past N months. But, you know, if you just all of a sudden apply this to the enterprise space, or to adjusting news feeds, or a zillion other things that you can think of — like, figuring out where it goes wrong, like, that’s actually really critical to a business. Like, you can’t put a product out there if — pick our sales forecasting example, and then it’s wrong, like, you can’t just go, “Whew, the machines make mistakes just like people.” Your VP of sales is kinda messed up too. Because nobody pays money to, like, a computer for it to be wrong. And so how do we think about that?

      Frank: Yeah, this whole idea is, sort of, transparency behind these models. In other words, do we know why they’re behaving the way they’re behaving?

      Sonal: Demystifying the black box.

      Frank: You know, a super active area of research right now, right? Which is, how do I make the deep learning models more transparent so that I can debug them, I can verify them, I can make sure there’s no systematic bias in them, right? Because until that, you couldn’t do important things like, “Hey, can this person have a loan or not?” Because the government will say you cannot make that decision unless we understand why it is that you’re making that decision.

      Sonal: But you’ve made the argument, Frank, that, you know, when it comes to, say, self-driving cars, we kind of — it’s no better or different than what the human mind does. We don’t know we can’t interrogate the black box of the human mind that’s driving that car. So, the counterpoint of that is that, well, you’re right, Steven, that, you know, you’re paying for this computer to be smarter. But the reality also is, this stuff is not that smarter than humans anyway, so who cares?

      Steven: Right. But the problem is first, that’s everybody else, not me. Yeah, like, I’m the best driver on the road. It’s all the other people. I mean, you know, I like to always go back to this — the wonderful, wonderful research on computers and society that Stanford professors Nass and Reeves did, because one of the things that they really realized, really back in the early ’80s…

      Sonal: Oh, Clifford Nass. Oh, all right, rest in peace.

      Frank: And Byron Reeves.

      Steven: And Byron Reeves, which was that there’s something about a mechanical device that produces answers that makes the human brain ascribe way more authority to it than there necessarily should be.

      Sonal: In fact, they even apply this to the world of voice recognition systems and the interaction.

      Steven: They applied it to voice recognition, they applied it to chatbots. Before they were chatbots. I mean, so the interesting thing is that I don’t know how — it’s going to take a major change in society for people collectively to just go, “Ah, people make mistakes.” And that’s okay. But machines can’t. Like, we have, especially in the United States, a very, very low tolerance for devices making mistakes.

      Sonal: Except when machines are doing a kind of alien intelligence that humans cannot do. Because, again, the most fascinating thing about our last talk about AlphaGo, this current talk — is that, at the end of the day, even though it opened — the AlphaGo Zero opened and closed with similar moves to what humans would do, it converged very quickly — there was a whole set of things in the middle that it did that were just things that humans would never have done. It’s actually, then, augmenting us in a very different way, because it’s adding a completely foreign intelligence. So, it’s not even comparable to our own to just judge…

      Frank: Yeah, it’s a different type of intelligence, in the same way that animals have a different type of intelligence. And then, you make all kinds of category errors when you say giraffes aren’t intelligent, or bats aren’t intelligent, they’re just intelligent in a different way.

      Sonal: Exactly. And we’re more for…

      Frank: We can’t use the human yardstick to compare them.

      Sonal: It’s the reason we are more forgiving when a dog pees all over your couch and your five-year old kid does the same freaking thing.

      Steven: That’s definitely the case. The challenge that we have in technology is, just, the perception. Like, I mean, you can just see it in the discussion of news feeds, and algorithms, and how people are, like, they should just be right. I mean, like, and people should just have debugged this before it all happened. And actually, it’s not even all that crazy sophisticated, what’s going on. And, what’s weird is, of course, at the same time people are critical of what’s on the front page of newspapers, or what’s in the first five minutes of TV newscast — which is literally the same decision made by a human being, who’s just deciding what should we show first on CNN versus Fox News. Some human just made up their — with some black box in their brain, which is augmented by their title of, “in charge of production.” And yet infinitely forgiving of those choices. And so, I actually think that there’s a lot here. And I don’t have an answer, but I think it’s no different than any other software. Which is, if you’re going to make something and offer it to people in a commerce situation…

      Sonal: It better damn work. And it better not be wrong.

      Steven: It better be clear, like, how it does it to you. I mean, like — people, like — Excel was really great at doing math very quickly. And then, one day, I find myself at the Naval War College having to explain myself to a bunch of generals. Like, how do we know it’s right. I literally just had to sit there going, “I mean, it’s just right.”

      Sonal: It just works.

      Steven: And those are the moments that contribute to me feeling, like, a lot of empathy for what’s going on in the marketplace now about this, and why I’m so alarmed — not alarmed, that’s totally the wrong word — so focused on this, you know, know the outcome. Because, like, until you’ve just sat there with a bunch of people who control nuclear weapons, telling you we’re using a spreadsheet to calculate it, the weight of being right doesn’t really hit you. Because before that, we were just like, “Oh, my God, it doesn’t seem to make mistakes.” Then we were, like, super ecstatic. And it made, “Look, the charts are cool. Yay, ship it.” And then one day, they’re like, “Is it gonna work or not work?” You’re like, we couldn’t prove it. Like, that was the essence of it. Like, I couldn’t go to people at Boeing, or people at the Navy or wherever, and Wall Street, and prove that Excel works.

      And then, of course, 30 years later, like, every time there’s a mistake in Excel, like, it’s a mistake in the human that typed it. Like, Ken Rogoff did a bunch of economic predictions that forecast the recession in 2008. And then, all of a sudden you find out his model was wrong, but the recession happened anyway. Did he make the right prediction or not the right prediction? And, like, how does that work? And I think that’s what’s going on here too, with these things.

      Potential bias in algorithms

      Sonal: I have a philosophical question, then, for you guys.

      Steven: That wasn’t philosophical enough.

      Sonal: Well, one more philosophical. We’re being very philosophical here. You know, you said something about how people make judgment calls for what news to show on television, etc. And we have these expectations about algorithms. And one of the topics we’ve discussed on this podcast — Frank, you and I discussed with Fei Fei — is this idea of bias in algorithms, and how algorithms, by definition, can be biased — is one possibility of this type of work. Because they kept using the phrase tabula rasa, which of course I find so fascinating, because in human development, there’s an analogous world of this, where there was this theory that the human brain was also a tabula rasa or blank slate. And then they quickly learned, like, “No, we have millions of years of evolution and DNA, that’s actually — [we’re] actually coming in inheriting things.” Is there now a possibility that algorithms can write themselves, [in] a true tabula rasa-like way, given this type of work? Is that just way out there?

      Steven: Oh, that one’s above my paygrade.

      Sonal: These guys are throwing up their hands for our listeners, by the way.

      Frank: I will give an example of, sort of, the things that are hardwired in your DNA. So, they’ve done a bunch of experiments that if you watch somebody’s hand getting pricked, your muscles in your own hand will involuntarily contract. Now, the big caveat is, if that person has a different skin color than you, your hand doesn’t contract.

      Sonal: Really? I didn’t know that.

      Frank: In other words, there’s something going on in your lizard brain that says, “That’s an ‘other’s’ hand.”

      Sonal: Primal

      Frank: Therefore, it’s not relevant to me. Therefore, whatever reflexes caused your hands to contract in sympathy, when somebody jammed a needle into that hand, that’s an example of this, sort of, prewiring. You know, the question is, can algorithms be prewired that way? I mean, you could…

      Sonal: Or un-prewired even.

      Frank: Or un-prewired.

      Sonal: Because that to me is where the opportunity lies.

      Frank: I mean, you could definitely write a set of rules that said, “You know, treat other people in a different way,” right? That would be top down. The peril that most people talk about today is not so much that the algorithms are biased, but you’ve fed it not enough data so that your prediction is biased. So, the classic example of this is, in the early days of vision recognition. Some of the image classification algorithms were categorizing people with dark skins as gorillas, that happened because they didn’t feed it enough data of dark-skinned people. So, when people talk about bias in algorithms, they are mostly talking about this phenomenon…

      Sonal: Limited by data.

      Frank: The human researcher or the human programmer selected an incomplete data set, and therefore you got biased results, as opposed to somehow the architecture of the neural network is biased inherently.

      Steven: And I think that that’s a very important point that is being well studied, it’s well articulated. But particularly in the supervised learning case, the data that’s input, at least today — almost every data set that you have is going to be and have some inherent bias, because you weren’t aware of these factors when it was being collected in the first place. Because I think it’s fair to say, the awareness of all of these issues is at an all-time high relative to that. But then again, you look at all the medical studies, and you’re like, “Well, there haven’t been very many women in most of these studies, or there have been only women but studying a drug developed by…”

      Sonal: That is true in biology and genetics research too. There’s a lot of limitations. Right.

      Steven: But I do think that anybody today embarking on using supervised learning, whether for all or part of the solution — that data set is implicitly challenged, and in particular, it’s the labels. Because even though which labels you pick or which labels you omit, it’s going to create some bias in the model that you’re unaware of.

      Frank: That’s exactly right. Your cultural background — your history, the way you grew up — will lead you to label an image a certain way. That may be different than somebody who — right? So what is the ground truth?

      Steven: Well, the best example of this or just looking at images and, like, sentiment analysis is such a big thing. But images — like, facial expressions — have just been studied all around the world for decades, multiple decades, about, like, what is happy, what is sad, what is questionable? The same with speech and intonation, like, sometimes, you know, ending on a high note sounds like you’re asking a question, unless you’re in another culture where that’s making an exclamation.

      Sonal: Or it can be vocal fry, and people are complaining that women shouldn’t speak that way.

      Steven: Well, the vocal fry thing is a perfect example of that going on right now. And actually, the high note was the — sort of the ’80s version of that same thing.

      Sonal: It goes in Vogue, too, right?

      Steven: And be super careful about the labels. And that’s — to Frank’s point, that’s a great place for transparency. Because if you can go to a future or a potential customer and talk about, “This is what our model is based on when we’re forecasting your sales,” or “telling you to optimize your assembly line.” That could really matter to that process.

      Advice for entrepreneurs

      Sonal: So, high level takeaways. Hybrid works. Always — this is a thing — this is, like, a refrain. I feel like I should get you a T-shirt that says, “Hybrid works, God damn it.”

      Steven: Well, be careful, because I’m really against a lot of hybrids.

      Sonal: Oh, yeah. Like, hybrid cloud computing.

      Steven: Yeah, hybrid in terms of solutions between old and new for coding. The old stuff that works definitely does work.

      Sonal: Old and new, academic and industrial, like, exactly — all the things that make it practical versus pure, so to speak, purist.

      Frank: I think another big takeaway of this is this “revenge of the algorithms” moment. Where there’s so much momentum right now that says, basically, we’re one labelled data set away from glory. Right? And this result basically shows you, “Wow, there’s a lot of mileage that you can get out of reinforcement learning, where there’s no data, no labels.”

      Sonal: One take away from me is just the element of surprise, because I’ve been thinking a lot, even just, you know, how — I mean, I used to work in the world of how humans learn. But what’s fascinating to me is that the system played itself at a level calibrated to itself at every level. And in human learning, that doesn’t happen. You’re taught by your parents, you learn from, you know, adults, you learn from people who have more experience. There’s all these different things that happen.

      And the thing that’s so fascinating to me, is that to me, it is very evolution-like. It’s like a big bang moment. And while I wouldn’t hype and say that means we’re going to end up here, I do think that’s very amazing, especially when you think about the surprises. Like, in the paper, one of the things they talk about is that the system learned something that’s actually very easy for humans to learn, way later in the game. It’s actually, I think, like, I forgot — shicho, or some specific move to, like, the ladder. The fact that the system took forever to learn something that’s very first for humans, I just — I’m endlessly fascinated by this relationship between humans and computers, and what we can learn from computers and vice versa. And also, what it means for the field, when you can actually add to how people learn to the field of artificial intelligence, machine learning.

      Frank: Yeah, with reinforcement learning, one of the challenges has been, sometimes you get into this training epic where there’s no improvement, right? Because you’re just playing these games over and over again. And, like, what is it that causes the next game to be smarter? And this one was, like, in three days, it got incredibly smart. So, yeah, hats off to these guys.

      Sonal: Yes. And let’s also add the other takeaway, which is that you can do amazing things with simple architectures, because we’re at a point and a moment right now where you can have 4 TPUs instead of 48.

      Steven: For me, the one of the takeaways is, if you can frame your problem with a set of constraints and a set of rules and a fixed set of operations, that’s a very powerful concept that changes — because we’ve been so data centric, people have stopped trying to think of their solutions algorithmically. And it’s entirely possible that there is an algorithm, which is the same thing that I think we would have said a year ago, which is — everybody was so focused on machine learning that traditional algorithmic approaches might have been overlooked.

      Sonal: And now…

      Steven: And here we are again with proof. But still, now we have to go back and say, “Well, you know, there’s some basic machine learning that can work. There’s some basic algorithmic stuff. But that key is really the set of constraints.”

      Frank: Yeah. For people who are interested, I highly recommend Andrej Kaparthy’s YC talk on this. You can search for it. It’s on Google Slides.

      Sonal: He works at Google, right?

      Frank: The whole talk is basically about where will artificial general intelligence come from? And he basically compares and contrasts this, sort of, rule-based world of Go, to something messy — like how would you build a pick and pack robot for Amazon? And so, if you’re interested in this topic of what’s the difference between rule-based board games, versus the messy real world, it’s a great presentation…

      Sonal: That’s great for the generalized intelligence side. Are there any parting messages for entrepreneurs building companies that have very specialized things? Because one thing we have argued in this podcast, including you, Steven, when we did a podcast on building product with machine learning — is that sometimes the best places to play are very specific domain focus, whether or not they have a clearly explicit set of rules or not. So, any thoughts on that?

      Frank: Yeah, so one takeaway is, one of the ways we evaluate startups from an investor’s point of view is, have you picked the best techniques to solve the business problem that you’re trying to solve? And I think this paper basically opens up a frontier of exploration that you might not have thought about before. Because, I think if you started an AI-centric company today, you would definitely be on the “get data, get labels, train model, make prediction” path. And this opens up another area you can try and figure out, “Am I solving a business problem where this supervised learning approach is gonna lead me to glory, instead of these — or, this reinforcement learning approach — is going to lead me to glory, as opposed to supervised learning or unsupervised learning?”

      Steven: I think [that] the most interesting thing for me in looking at different companies and just talking to founders is just that there’s this world going on of advances and new things all the time. And you could get on the treadmill of always trying to be the newest thing. And, you know, we knew that the next generation of companies, you know, two years ago, we’re just gonna be, you know, whatever you were doing before plus AI. But the thing was, it wasn’t clear that that was always the best solution. And then, you replaced the AI with “now it has to be data and machine learning and labeling.” And so what Frank was saying is just super important to internalize, which is that part of being a founder, and building a new product is knowing that the reason why you’re choosing the technologies you’re choosing — and not just because you think that’s what investors are looking for. Our jobs, on that side, are to actually ferret out, like, who actually has a handle on the problem they’re solving, and a line that they can draw from that problem to their chosen solution path. And that is too often overlooked.

      Frank: That’s exactly right. My favorite joke that I tell is — when I ask entrepreneurs, what machine learning algorithms they’re using in their product, when they’ve claimed to be a machine-learning-based startup. The answer I am not looking for is, “I don’t know. I’m sure we’re using the good ones.”

      Sonal: Well, on that note, you guys, thank you for joining the “a16z Podcast.”

      Steven: Thank you.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      • Steven Sinofsky is a board partner at a16z. Prior, he spent 22 years at Microsoft in a variety of positions including leading the Windows division.

      Adjusting to Trade … and Innovation

      Russ Roberts, Noah Smith, and Sonal Chokshi

      Beyond the overly simplistic framing of trade as “good” or “bad” — by politicians, by Econ 101 — why is the topic of trade (or rather, economies and people adjusting to trade) so damn hard? A big part of it has to do with not seeing the human side of trade, let alone the big picture across time and place… as is true for many tech innovations, too.

      Speaking of: how does the concept of “trade” fit with “innovation”, exactly? They’re both about getting more from less — as well as creating new opportunities — shares Russ Roberts, host of the popular EconTalk podcast (and fellow at Stanford University’s Hoover Institution, PhD in economics). But there’s another very provocative theory at play here — fast-forwarding us from the time of the Industrial Revolution to the 2000s — that could make us rethink the relationship between trade, capital, labor, productivity/economic growth, shares Noah Smith, columnist at Bloomberg View (and former professor of finance at Stony Brook University, PhD in economics).

      And where does China come in — and out — of this picture? Put it all together, and maybe, just maybe, it could help explain why we’re investing in labor-saving innovations/ automation more than ever today. Because one thing is for sure, agree both Roberts and Smith — who otherwise argue with each other on this episode of the a16z Podcast (with Sonal Chokshi) — you can’t stop the march of technology. It’s here, it’s coming, and we’re just going to have to meet it, prepare for it, …roll with it.

      Show Notes

      • Discussion of the history of trade, including the Industrial Revolution [0:44]
      • How trade and technological innovation are linked [9:28]
      • The impact of trade between rich and poor countries [16:53], and how trade with China affected both economies [26:13]
      • Trade deficits, and a debate over their effects [31:17]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today, we’re talking about all things trade and innovation. And it’s a really meaty topic. So we begin by talking about why, beyond the obvious, the topic is so hard, what trade is responsible for when it comes to tech change and jobs. And we cover everything from some really interesting theories of the past, and whether they matter for the present and the future, to the debates around productivity, manufacturing, China, protectionism, and more. So a hell of a lot.

      To have this conversation, we have two special guests. Russ Roberts, host of the longstanding and excellent “EconTalk Podcast” and research fellow at Stanford University’s Hoover Institution. And Noah Smith, who did his Ph.D in economics at the University of Michigan, was a finance professor at Stony Brook University, and is now a columnist at “Bloomberg View.” And he’s the first voice you’ll hear.

      Causes of the Industrial Revolution

      Noah: We’re really talking about the part of trade they don’t teach in Econ 101. We’re talking about the adjustment to trade. And in Econ 101, there’s, pretty much, this unstated assumption that economies adjust very frictionlessly and very easily, and everything goes very smoothly. To be honest, it’s really not easy to tell when that will and won’t happen. And there’s no general theory that economists have about adjustment. Economists often think to the very end and think about equilibrium, and ignore the path for how you get there. But, you know, that can be a little bit like saying, like, World War II is great because it brought us human rights. But the road to get to the UN Human Rights Convention was a little bumpy and…

      Sonal: Yeah, the reality is very messy in that way.

      Russ: The only thing I would add to that. I think a good Econ 101 teacher talks about the adjustment. I think it’s a bad teacher and a bad economist who says, “Well, trade creates net benefits, so therefore it’s good for the country.” Trade is good on average for the Americans, and it obviously has big distributional effects depending on who you are, what you do. There could be long-term distributional effects, that your skill is no longer valuable. And there could be a short-term adjustment which is, it’s no longer valuable. You’re going to have to find something new to do and it may take you a bit of time to find it. And it takes a very long time.

      Noah: Right. Or you may never do it and you may just retire, go on welfare, and then someone younger may then find something better when they get out of school.

      Russ: Correct.

      Sonal: I don’t want to go into Econ 101, but I’d love us to actually have some building blocks for the conversation, simply because there’s a lack of economics education. If you guys were to, sort of, launch off with what we need to know about trade, like, what would be some of the big ideas that we should start with?

      Russ: Well, trade as a political issue goes back shockingly long. I’d say it goes back probably about 800 years.

      Sonal: Why not longer?

      Russ: Well, around 800 or 900 years ago.

      Sonal: I’m pretty sure people have been trading for centuries.

      Noah: There was probably someone in Egypt saying, like, “Oh my God, this Mesopotamian grain is going to put us all out of business.”

      Sonal: Yeah.

      Russ: For sure. But the issue was whether trade was good for the country or not. It’s still the issue. But around 1776, Adam Smith came along and said, “We’ve made some fundamental mistakes in mercantilism.” Mercantilism was this idea that trade could harm you because you’d lose your gold. And Smith pointed out gold is not something you can eat, it’s not really the measure of a nation’s well-being or people’s well-being. And he made the case for the benefits of trade. 

      And if we come to the modern times, I think most economists would say trade is important because it allows us to get more from less, just like innovation does. It’s a way to increase productivity. So instead of making things for yourself, you trade for the things that you don’t make as well as somebody else, as cheaply as somebody else. So…

      Sonal: There’s actually a really good anecdote that I saw about this on Twitter recently. And I don’t know if it happened recently or not. But, basically, some guy in the last few years decided to make a sandwich from scratch. And it was everything from growing the vegetables, to milking the cow, to flying to the ocean to make salt. And, by the way, he didn’t even include that flight in this price or it would have been more, but it cost him $1,500. You know, that’s way more expensive than, like, $7 which is the average price of a sandwich.

      Russ: If you tried to make everything for yourself, true self-sufficiency, you’d probably starve to death. In modern times, though, we worry about, and correctly so, that some of the people who benefit from trade are not necessarily going to be the same people who bear the costs of trade. We make something abroad, it means we’re not buying it here domestically anymore, and those people are going to have to find new ways of working. Trade makes things inexpensive for everybody. That’s great. It also makes it harder for some people to use their skills. The part that I always like to emphasize that people forget, is that when you make things cheaper for everybody, that frees up resources to make new things you wouldn’t have had before. And the last thing, I would argue, is that the biggest benefits from trade take place over longish periods of time. The opportunity for the next generation to come along and use its skills well. But it can be hard on this generation.

      Sonal: The one thing economists seem to almost universally agree on is this — I mean, there’s never any universals with economists. But, like, this idea that it’s actually in your best interest as a state, country, whatever, to open up your borders to trade. Politics aside, it seems to be one of the fundamental beliefs.

      Noah: Just because economists, you know, express agreement on something does not mean it’s right. But in this case, I think that, broadly, the economists’ consensus is fairly correct. But then there are some important caveats that get lost when we talk about that. And one of the things I think is important to emphasize is our ignorance. We understand some basic things about trade, and then there are probably some things about trade we don’t understand at all. So for example, if you look in history, a lot of people wonder about how trade affected the industrial revolution. As Russ said, trade can look a lot like innovation. In fact, in economic models, the gains from trade will look just like productivity, and there’s a complete equivalence between those in a lot of models. So, basically, if you have a robot to do something cheaply, you know, from the consumer’s perspective, that’s exactly the same as getting a low-paid worker in China to do it cheaply, so trade can be like innovation. But when we look at the really big epic booms in innovation, like the industrial revolution, trade may have some more complex effects on that.

      Sonal: Yeah, we actually had economist Joel Mokyr on the podcast talking about the industrial revolution, but it was more about the open trade routine ideas. And how that led to what he called an unprecedented quantum leap in economic and tech progress. And included things like the steam engine and automating fiber looms.

      Noah: Well, one of the big theories about the industrial revolution is that a combination of expensive labor in Britain and the Netherlands, and cheap resources from the Western hemisphere — the so-called New World — combined to make it suddenly worth entrepreneurs’ while to start using machine labor instead of humans. And the Romans had all the technology for steam engines, they even did some mining with steam. A Roman inventor had a little steam engine on his desk called the aeolipile. But they never really deployed it en masse. And China had all the technology for power looms and just didn’t…

      Sonal: Yeah.

      Noah: …put it together and start using it.

      Sonal: Even in India too. And so these things go way back. So your point is that this wealth of the New World…

      Noah: Right, the cheap capital, essentially, from the New World — cheap resources and capital, combined with expensive labor in Britain and the Netherlands. There is this theory that that’s what sparked the industrial revolution, by causing business owners to finally say, “All right. You know what? Forget you, humans, we’re just going to use a bunch of machines.” And then suddenly they said, “Wait a second, these machines are really good. Maybe we could make slightly better machines.” And tinkered and tinkered. And then you got, suddenly, on this track of continuous improvement and continuous searching for new industrial technologies to make people richer. And, obviously, that is the greatest episode of human flourishing in the world. By far the biggest event. Probably bigger than the agricultural revolution.

      Russ: You can almost look at that as we flipped a switch and something just…

      Noah: Something happens.

      Russ: A lot of things exploded. Our lives are remarkably easier, safer, longer, higher quality in so many dimensions — not just monetary ones, and it’s always important to add that. And yet at the time, of course, the political economy of that revolution made Dickens famous. Gave Dickens something to write about — the costs of that in human toll and human lives — misery, in the short run, was very strong. A lot of people were put out of work by those machines, a lot of people had low wages as a result of those machines. And a lot of people worked with those machines and didn’t make a lot of money, and had a very dispiriting existence. Charlie Chaplin’s “Modern Times” being the best pop culture version of what that experience was like. The artisans’ life, the craftsmanship of previous days, and the meaningfulness that people got from their work suddenly changed. And the point I want to emphasize is that, that response — the anger and fear that that engendered — was very natural, very real, very justified at that point. And I think it’s important, though, to look ahead to future generations and say, even though they were very unhappy at the time and wanted it stopped, those innovations, are they happy now with the way the story turned out? That their grandchildren or great-grandchildren…

      Sonal: That they could see. Fast forward…

      Russ: …had these remarkable lives.

      Sonal: …into the future. I actually remember you talking about this in “The Human Side of Trade,” where, like, 40% of the workforce in the 1900s was in farming, and now it’s, like, 20%. So if you go back in time, that farmer would have been really crying and sad about the lack of opportunity for himself and his kids. But if you brought that farmer into the present, he’d actually cry in happiness, to see his great grandkids have so much more than he did or they did when they were going through the really hard times. But it’s obviously really hard to see that big picture right now.

      Russ: It’s hard to do that. And I think it’s important to do that, because economic change is dynamic by definition, and it’s lively, and it’s hard to know what’s going to happen. But often, the longer run impact is very different than the immediate impact.

      Trade and innovation

      Noah: Absolutely. Not just [in the] long run, but I think it’s important to emphasize the nonlinear impact. The fact that sometimes small things can have big consequences. But now, back to the theory. So, one theory of the industrial revolution is that it happened because, basically, business people started replacing overpriced British and Dutch workers with robots.

      Sonal: Yeah.

      Noah: The robots of the day. And the question is, what if, in a much smaller way, in the 2000s, we started replacing robots with cheap workers in China? What if we were going the opposite direction? And what if trade can temporarily, under certain circumstances, have a slowing effect on innovation? I mean, it’s obviously not going to stop the industrial revolution from happening, or something like that. We’re not talking about effects that big anymore. But what if trading with, you know, the New World, so to speak, with all its abundant resources, made capital really cheap and caused a tech boom? And what if the, sort of, productivity slowdown that we saw starting in the early to mid-2000s, actually — what if part of that came from relying on outsourced labor in that boom, instead of on better machines?

      Sonal: Why is that idea important? Like, what’s the natural implication of that?

      Noah: The natural implication of this is that, basically, the more cheap labor you get, the less incentive there is for labor-saving innovation, for entrepreneurs and technologists to invest in labor saving innovation. And I don’t necessarily endorse this theory. It is an idea.

      Russ: It’s a clever idea. And I think there’s some truth to it. The reason I think it’s an important idea is that it suggests an explanation for the productivity slowdown that isn’t as alarming as the view that says, “We’ve figured everything out that we can, and everything is going to get worse from here on out.”

      Sonal: Yeah. Like the Robert Gordon view of things. Fall of growth, death of innovation.

      Noah: Maybe we took a one-decade pause in automation. And maybe now that wages have risen so much in China, and things like that, maybe now we’re going to get back to the task of automation.

      Sonal: Innovating. In this conversation, it’s basically machine improvements to save labor. That’s your definition of innovation in this context.

      Noah: Right. If it doesn’t save us labor, why are we doing it?

      Russ: Right. You would think that the opportunity to use inexpensive labor outside of the United States — a lot of it is from China, but it’s also lots of other places — should still lead to gains to Americans in the form of lower prices through the innovation that’s taking place. So in a way, if you think about what’s the difference between China closing its borders, and we never had this opportunity to trade with China. You’re suggesting we probably would have invested more in capital here that would have raised workers’ productivity here. Ironically, what happened instead, is that workers, you’re saying, got less access to capital, therefore didn’t get those productivity gains. But we, as consumers, still should have gotten the benefits of that in the form of the lower price, because that shouldn’t affect it?

      Noah: Exactly. And so, really, under some pretty simple conditions, you can show that as China opens itself up and we choose to outsource to China, the overall aggregate short-term gains will be bigger than they would’ve been if we had done the hard work of trying to invest in new machines. But, if you think about the long-term effects — so that’s talking about what we call, you know, static efficiency or short-term effects. You can show pretty easily that that’s going to be bigger. And that’s what Econ 101 always talks about, static efficiency. When I’m talking about…

      Sonal: Wait, what’s static efficiency, exactly?

      Noah: Static efficiency just means, like, how much cheap stuff are we getting today? How much productivity are we getting today?

      Russ: How much is the pie bigger than…

      Noah: How much is the pie bigger?

      Russ: …than it could be if we’d not had that opportunity?

      Noah: Exactly. The pie today, when we think back to the industrial revolution and the, sort of, long-term consequences. If you buy this theory, then for hundreds of years, entrepreneurs were doing the easy thing of getting a whole bunch of cheap medieval laborers to run their printing presses, and looms, and mines, and whatnot. And they were maximizing their static efficiency, their short-term productivity. But, because they didn’t do the hard work of invention and innovation, they didn’t get those knock-on effects further down the road. And when they finally said, “Okay. Oh, my God. Workers are so expensive we’re going to just spring for a steam engine.” Maybe in the short term, that cost them more. And so they wouldn’t have done it if they hadn’t had to. But in the long-term, that opened up whole new veins of innovation.

      Sonal: There’s actually a concrete example that just jumps into my mind about this right now playing out today, which is India’s decision, or discussion around not having driverless cars, because they don’t want to displace the labor that can — they don’t want to take people’s jobs away, essentially. And then my initial reaction is exactly what you’re talking about, which is — but they’re doing this at the cost of then investing in these labor-saving long-term innovations.

      Russ: That’s a fantastic example, because as Benedict Evans points out in an “EconTalk” podcast, one of the impacts of, you know, autonomous car introduction is a dramatic reduction in the cost of getting from here to there.

      Sonal: Yeah.

      Russ: So if you think about a current cab ride. A current cab ride or Uber has a huge labor component, and it also has an insurance component. But the cost of getting from here to there is going to go down. And that means that more people are going to go from here to there. Which means there’s going to be more opportunity and more resources available to do new things. So the worry is, and this is the essential trade political question. The worry is, are those new things going to be useful in employing the people who used to be the drivers of the cabs and the Uber’s? And that is the real question.

      I think we’ve politicized trade dramatically in the last few years. Is it just for the benefit [of] corporations, of the top 1%? And I think that’s totally wrong. However, there is a real issue, which is, will the people whose skills have been employed productively in, say, the cab or truck driving business — will they be fruitfully employed, beneficially employed — in whatever comes along? And new things will come along. That, I’m very confident about. What I’m not so confident about, and am concerned about, as, like, everyone should be, is whether the skills of the people who are displaced are going to be available.

      Sonal: So we’ve morphed into a discussion now or debate around tech and jobs.

      Russ: But it’s the exact same discussion that people should have about trade with China and displacing manufacturing jobs in the United States, or in innovation like the autonomous car displacing drivers.

      Sonal: Is it the same as the debates that have played out since the beginning of time, or is it actually different now because technology is accelerating faster? Because a big part of this is the adjustment time.

      Noah: The debates are always the same. That doesn’t mean that the underlying truth is the same. And sometimes we confuse this. So for example, anti-war movements were making broadly the same arguments during the Iraq war and during World War II. But those were obviously very different wars.

      Sonal: That’s a good analogy. So how does that play out in this question of trade?

      Noah: The people who were scared of trade are scared because of things that very immediately affect them.

      Sonal: Right.

      Noah: When you hear economic debates, almost everyone is talking about what immediately affects them — what they see in their own job, in their own industry.

      Sonal: Their day-to-day livelihood or their children. Maybe one generation out at the most.

      Noah: Exactly.

      Sonal: Yeah.

      Noah: And so you see people wondering, “Will I lose my job? Will my business face…”

      Sonal: But that’s old.

      Noah: “…increased competition?”

      Sonal: So I’m still not hearing what’s new. What’s the different underlying truth in this case?

      Trade between rich and poor countries

      Noah: What’s new is the type of trade, what will happen, who we’re trading with. What we’re trading, what kinds of trade we’re opening up. What the policy changes are, and what kind of other innovation technology is growing that alters this process. Those things can be very different. So, for example, opening up trade — the simple example — opening up trade with a rich country, like a country in Europe or, you know, Japan, for example — is in Econ 101 or any theory going to be very different than opening up trade with a very poor, low-wage country. So…

      Russ: Same with immigration.

      Noah: Same with immigration.

      Russ: High-skilled immigration is going to be very different. Both of which are good for the country as a whole, but have very different effects on particular groups…

      Noah: That’s absolutely right.

      Russ: …of people within the country, so that the politics are going to respond to that in different ways.

      Noah: Yeah. And so the point is that trade issues now, for example — people are talking about TPP and TTIP, which is basically a trade treaty with, you know, Japan and Korea, a trade treaty with Europe. Those are trade with rich countries, and that’s not going to look at all like opening up trade with China did in the 2000s. It is not going to resemble it at all, and yet people treat it as if it’s the same thing. One interesting thing is, if you look at some of the research papers that are coming out now like Autor and his co-authors.

      Sonal: Autor et al.

      Noah: Autor et al.

      Sonal: If I recall, he’s arguing now that some of the benefits of trade with China actually were not as great as he thought they were, and that we could have, in fact, slowed down our process of trading with them. As, kind of, the gist of what the latest paper was.

      Noah: Right. So, what’s interesting is that during the ’80s and ’90s, people were very scared of trade with Japan and Germany. You know, Japanese car companies coming in and out-competing our companies, and blah, blah, blah.

      Sonal: Yeah.

      Noah: And so what I’m saying is that when economists around in the late ’90s and early 2000s went back and looked at those episodes, what they found is that the displaced workers almost all found new jobs in doing very similar things for the same or higher pay. If you look just at manufacturing. It affects more than manufacturing. But if you look just at manufacturing, manufacturing employment really held up very robustly during all of that so-called Japanese and German competition, all throughout the end of the ’90s. You know, very robust, and you had this very smooth adjustment. And people seeing that we’re like, “Okay, safe.”

      Then China. First, we had most favored nation trading status, permanent normal trade relations, and then the WTO entry. And what Autor, Dorn, and Hanson found is that the adjustment of the average worker in competition with Chinese workers was much, much slower and worse. And they found that, you know, the average displaced manufacturing worker in the ’80s and ’90s who was displaced by Japanese competition tended to find just as good or better a job. The average manufacturing worker displaced by Chinese competition in the 2000s got a crappy low-paid service job, or went on welfare, or disability — which, in some cases, has acted as a shadow welfare system. So people who drew the lesson from the ’80s and ’90s that the economy always adjusts real quickly, and that trade is essentially safe, didn’t think very carefully about the difference between trading with other rich countries and trading with a labor-rich capital poor country.

      Sonal: Like China and India?

      Noah: Like China, and maybe like India. Although, there’s some difficulties with infrastructure in India that means that India’s unit labor costs may still be high. So when we talk about cheap labor, one thing to mention is we don’t just mean wages, we actually mean unit labor costs. Which is basically, for a given worker, how much can I produce? And so, China’s labor was cheap not just because Chinese wages were low, but because Chinese wages were low relative to Chinese productivity. Indian productivity is held back by the fact that electricity keeps going out. And it’s very hard to get the products on the roads — the infrastructure sucks. So when you have an Indian factory on the coast, Indian unit labor costs are lower than Chinese unit labor costs. The reasons China was cheap — I mean, cheap wages were part of it, but they were subsidizing energy, they were giving everybody really cheap coal without worrying about the environmental impacts. They had very low labor standards. And they had a bank finance system that essentially fed super cheap capital, super low interest loans, to all these Chinese enterprises.

      Sonal: Overall lower unit labor costs.

      Noah: Overall lower unit labor costs. And it was so low, and so sudden, and China is just so big. I mean, if we traded with a tiny little poor country, we wouldn’t notice anything, but China is four times the size of our population. And trading with that sudden monster of a labor dump being dumped on our markets had a very deleterious impact on the lives of a lot of the workers that it hit in America.

      Russ: So here is the problem I have with that analysis. There’s, obviously, some truth to it. I think the trade shock from China is different from the trade shock from Japan or Mexico. And I think they’re probably right that the adjustment for workers in the 2000s was tougher than the adjustment for workers in the ’80s. That it was much harder. I do think it’s part of a very long trend, though, that goes back to 1945. The aftermath of World War II, manufacturing employment steadily left the United States. It just got a little quicker in the ’80s, and then quicker still in the 2000s as a proportion of total employment. So the question is, for me, what did we learn from this? And is it because they are different kinds of countries?

      Sonal: Yeah.

      Russ: Is it the speed of the adjustment that we’re struggling with? Or is it a third factor, which I want to put on the table, which is that the labor market — I don’t think works as effectively as it worked in the ’80s and the ’90s. People are having a lot harder time moving to find new work, and we don’t fully understand why. There’s different theories. Is it because the jobs are in the cities and rents in the cities are very expensive? I don’t think we understand why. And if we don’t understand why, we’re going to have a very tough time doing the right kinds of policies to adjust to it.

      Sonal: Exactly.

      Russ: No matter what, it’s certainly true that we need a better system to help people find new opportunities, right? And give them the skills that they need to find new employment. But we don’t really have a good understanding of what the nature is of the problem we’re trying to fix. And the reason that’s important, and this is, again, where it ties into technology and autonomous cars, artificial intelligence, robots, etc. The worry among different people, and I think it’s a genuine worry, is that in the past, when you were displaced by technology, you’d find a new opportunity that the new technology helped facilitate, because it made things less expensive. But what if the technology takes over everything, and there’s no place left for people’s skills?

      Noah: That’s a really important question, but let’s go back to a couple points. Manufacturing employment shrank as a percent of the total workforce since 1945, but total manufacturing employment didn’t.

      Russ: We had a growing population.

      Noah: And total manufacturing employment has shrunk since 2000 dramatically. So, at the exact same time China entered the WTO, total manufacturing employment in the United States — which had been at a stable level for decades and decades — starts to plunge?

      Russ: We also, at the same time, had the introduction of the computer and the internet.

      Noah: That was in the ’90s. And the big investment boom for computers was in the ’90s. And that’s when we got word processors, and spreadsheets, and blah, blah, blah. We got broadband in the late ’90s.

      Russ: Yeah, we also got the application of those techniques.

      Noah: It absolutely could be. What I’m saying is, the common notion that manufacturing employment steadily declined is wrong. What actually happened was that manufacturing declined as a percentage of our economy, because most of the new things we wanted from the new economy — new technologies — were services. Basically, we got a whole bunch of manufactured stuff, and the amount of physical stuff that we bought stopped increasing very fast, but we kept employing about the same number of people, and their productivity increased. But the percentage of manufacturing in the economy went down because our demand for services increased much faster after, you know, the 1960s than our demand for manufactured stuff. 

      But it wasn’t until the 2000s that absolute manufacturing employment started to fall, and that manufacturing employment started to fall along with manufacturing productivity slowing down. So you saw a productivity slowdown at the same time as a fall in employment. When you see the trend in manufacturing worker productivity suddenly get flatter, and yet manufacturing employment plunge off a cliff, in absolute terms, not just relative terms, for the first time in many, many decades — you’ve got to wonder. If this unprecedented decrease in the absolute level of manufacturing employment was from automation, why did it coincide with the productivity slowdown? Why wouldn’t we see a productivity speed-up? If the manufacturing employment started plunging off a cliff because of technology, why don’t we see a productivity speed-up in that industry, because instead we see a slowdown?

      Russ: The only thing I’m confident of is that it’s tangled together, there’s a bunch of things going on at the same time. I agree that it’s possible that the dominant effect is trade rather than technology. I also would suggest it’s really hard to measure productivity and output in this area, because we’re agglomerating…

      Noah: It’s a lot easier in manufacturing than in other industries…

      Russ: It is. It doesn’t mean it’s easy.

      Noah: …because you can count the actual things.

      Russ: Yeah. But that’s not enough. You can’t count the things because you’ve got to aggregate computers, airplanes, bolts, screws, nails, and other things. So you’ve got to use a dollar value. And then all of a sudden, you’ve got issues of how do you aggregate?

      Noah: Within industry productivity, you can measure, like, number of planes.

      Russ: Right. That’s…

      Noah: You’re not aggregating by price there.

      Russ: Correct.

      Impact of trade with China

      Sonal: We’re talking about the manufacturing and the productivity slowdown. But where does this idea of people being able to buy more because of the cheap products that are coming into the market fit in? Because while people may have had a hard time finding jobs, they can buy more with less.

      Noah: We do that by trying to adjust for inflation. So, if we have very slow inflation, prices aren’t going up that much relative to wages. What we’ll see is what we call real wage gains. If…

      Sonal: You get more money for…

      Noah: …you get more stuff.

      Sonal: …for your money. Right.

      Noah: More money for your buck, then your real wage is going up. It turns out that real wages in the United States have been pretty close to flat. Now, they’ve risen the last couple years, but in the 2000s, real wage gains were very, very anemic. Here is where it gets tricky. There’s many different measures you can use for inflation, so how much should the price of healthcare weigh in that? How much should the price of rent weigh in that versus the price of the TV? It’s very tricky. And you can use some inflation adjustments that show a pretty strongly rising real wage, or that show essentially no wage slowdown in the 2000s.

      Sonal: Right.

      Noah: Or you can use PCE, inflation shows that one.

      Russ: The PCE. But the other problem, of course, is quality. So, weave in something like a TV. A TV today is very different from a TV 15 years ago. Incredibly different from a TV of 30 years ago. The number of things that your phone does and replaces is thousands of dollars’ worth of technology. I have a video camera, I’ve got a still camera. So when the phone gets more expensive in nominal terms, meaning the actual dollars you pay, or a little bit cheaper sometimes, but it does a thousand more things or holds a thousand times more songs…

      Sonal: It’s like a category error, right?

      Noah: Right.

      Russ: …you need to adjust for that and the Bureau of Labor and Statistics isn’t particularly good at adjusting for it. So that’s another caveat you have to put on the real wage issue.

      Sonal: But what do you make of this argument about, just going back to the theme of trade in connection to this, and the entry of China and what that did for our economy? When I think of the reverse, that you could, say, limit trade — and this is like the protectionist argument that comes up sometimes. I think about it disproportionately affecting poor people, because they’re the ones who go to Walmart to buy the cheap products on the shelves.

      Russ: Well, if you don’t have any job at all, obviously, the fact that prices are cheaper is not so important to you. If you do have a job, even a job that pays less, but if prices are a lot lower, you could still be better off.

      Sonal: Right.

      Russ: And one way to think about Walmart, it’s a retail outlet for China.

      Sonal: That’s exactly how I would think about it.

      Russ: The thing that’s remarkable to me is that we are lucky right now to live in a time when the unemployment rate is incredibly low, it’s 4.3.

      Sonal: Didn’t we actually just hit a statistic, that we are actually finally have risen on the jobs? I just saw this this week.

      Russ: The employment rate, the actual percentage of people who are employed which is not…

      Noah: Employment to population ratio, the best labor market indicator.

      Russ: The best. Much more important than unemployment.

      Noah: Prime age, prime age.

      Russ: It’s finally getting better. And you emphasize prime age because a lot of the drop that we’ve seen is actually the aging of the population, not a crisis in the labor market.

      Sonal: Like what happened in Japan.

      Russ: Right.

      Noah: We could use age-adjusted prime age.

      Russ: Not going to disagree with the reality that for 10 years, we didn’t do very well. We had a horrible recession that was just a disaster. But now things are pretty good, and yet the perception of how the economy is going is, “It’s awful.”

      Noah: A lot of people are just thinking politically. When you ask them, “How is the economy?” They think, “Do I like the president?” I want to say something else. The thing about slow adjustment, about some workers, sort of, permanently losing their livelihood, the stuff about competing with low wage versus high wage countries, the thing about the possible innovation slowdown in automation from a decade of cheap labor. All these caveats, they’re all over. This is all about the past, and this is an argument about the past. The China shock is over. China unit labor costs are now at parity with the United States’ labor costs, and there is nothing on the horizon that looks anything like China. Basically, the China shock was this giant one-time thing. Suddenly, you had a fifth of humanity with very little capital and a huge amount of labor, and high productivity, and all this cheap financed energy, and capital, and stuff — be dumped on the world economy. A fifth of humanity in one decade.

      Sonal: Sticker shock, all at once, huge scale.

      Noah: Correct. It’s all at once. China’s no longer undervaluing its currency. That was very mercantilist. They were building up their U.S. dollars so, like, modern-day gold. They were doing that in the 2000s. They have stopped now, and have actually been propping up their currency instead of making it artificially cheap. They’re selling off that giant pile of reserves, they’re going into reverse mercantilist mode. So all of this is over. And so this argument is an academic argument for “How should economists think about trade, how should economists study trade, how should economists talk to the public about trade?” It is not an argument about “What should our trade policy be right now?” Trade has gotten less ambiguous a lot in this decade than it was last decade.

      Russ: My only comment is we focus a lot on our neighborhood, which is realistic and understandable. But I think it’s important to remember that trade has transformed lives elsewhere.

      Trade deficits and their effects

      Sonal: We didn’t talk about the argument of asymmetry in trade, and what happens when you have one country that’s being more protectionist and another one that’s not. I’ve heard the argument. I think Paul Krugman made it, or maybe Adam Smith, I don’t remember, but, “Even when you’re open and everyone else isn’t, everyone still benefits.” And that’s the bottom line of this free-trade idea. What happens when it is disproportionate?

      Russ: So, in the 1980s, when the debate was over whether we should let Japanese cars come into the United States to compete with American cars, a lot of people said, “We shouldn’t because they don’t let other products of ours into their market.” And my view was, “If they want to harm their own citizens by keeping out American products, that’s their choice. Don’t confuse that with the fact that we benefit when we let in Japanese cars.” There’s a mistake and focus on the value of exports. That, somehow, trade is only fair if we have balanced trade. That we import exactly the amount that they accept of our goods. It misunderstands the fact that trade in goods is not the only thing we do. We trade in money, and assets, in capital surplus, and services as well. And so, America, on average, is going to be better off when we open our borders. Not as much as it would be if other countries joined us and opened theirs, too. People say, “Yeah. But we need to use that as a negotiating tool.” And my view is, it doesn’t work very well historically. So if we say to China, “We’ll practice free trade only when you do.” They’ll say, “Okay, fine. We won’t, and you won’t.” And then we’ll end up in, not just a trade war but possibly a real war.

      Sonal: Right.

      Russ: So, I think trade is very important for that reason. It builds bonds between countries, even if other countries don’t want to be as open as we are, say, for political reasons. Just one more example. And the Japanese don’t let in rice, foreign rice, because the Japanese farmers are very small and very politically powerful.

      Sonal: Yeah, just like the corn farmers in the U.S. I mean, like, high-fructose modified corn starch or corn syrup. Whatever. You know what I’m talking about. 

      Russ: We pay a premium for sugar, because we keep out foreign sugar. People in China pay an enormous premium for rice. It’s just pure politics. And the world would be a better place, I believe, if we didn’t listen to that.

      Noah: So, that’s generally right. There are a couple caveats.

      Russ: Thank you, Noah.

      Noah: Thank you, Russ.

      Sonal: I like how you guys are fighting by caveating, it’s great.

      Noah: Well, caveating is important.

      Sonal: It’s very important.

      Russ: It’s the economics way of life sort of caveat.

      Sonal: It’s actually the “a16z Podcast” way, which is, I love nuance.

      Noah: So, to inject some nuance. The idea that trade builds bonds between countries, I believe, is an overrated idea. Because on the eve of World War I, Great Britain and Germany were each other’s largest trading partners. My friend Paul Poast, who is a political science professor, has shown that trade agreements are a great way of cementing alliances. But if you don’t have an alliance, then, you know, it’s not as helpful. The other caveat is that exporting at the company level raises productivity.

      Sonal: What do you mean by exporting at the company level? 

      Noah: So, when a company enters a foreign market, it experiences a jump in productivity. Obviously, companies that were more productive to start with are going to be more competitive in foreign markets. There’s pretty good evidence that exporting forces companies to up their game in general.

      Russ: Trade generally does that, right?

      Noah: Trade generally does that. So…

      Sonal: By the way, is that as true of services as it is of products?

      Russ: It’s harder to export services.

      Sonal: Right.

      Noah: Services, we do have a lot of service exports, actually.

      Russ: We have foreigners who come to America to go to college.

      Noah: Things like that.

      Russ: That’s an export.

      Sonal: Right.

      Noah: And our universities are competing with Chinese universities. The point is, if we have a large trade deficit, that’s not necessarily bad. Because, you know, if there’s unbalanced trade, someone has got to run the deficit. But if we have a large trade deficit, it means we’re not exporting as much as we would be if there were balanced trade. And that could be depriving some of our companies of this pressure of international competition that forces them to up their game.

      Sonal: It’s kind of similar to the argument you made earlier, actually, about just how it improves innovation overall. It’s actually in the same vein.

      Noah: Could. It could be in the same vein. You know, in most economic models, technology falls from the sky like manna from heaven, and companies just take the technology that the magic inventors somewhere invent, and they apply this.

      Sonal: It’s like the deus ex machina in, like, literature.

      Noah: Yes, it is. There’s an argument to be made that encouraging your companies, and not just your big companies like Boeing, but your small companies, to get out there and sell on the foreign market instead of just, sort of, cozily selling to, like, the neighbors in Ohio.

      Sonal: Their limited market. Right.

      Noah: But actually try to get out there and sell to people in Bangladesh or wherever.

      Sonal: Well, that’s where the internet comes in.

      Noah: But I worry that our companies are experiencing lower productivity from lack of exporting. Obviously, in the 2000s, the currency thing with China was part of that. But even then, it was a modest part. I think, really, the case is we have got this giant domestic market, and we run, you know, trade deficits. But we’ve also got this giant domestic market and it’s just really easy for companies to, kind of, slack off.

      Russ: Well, I disagree because I think politically that it’s going to end up with politically powerful companies getting more than just the incentive to export.

      Noah: I mean, small companies. I mean small companies.

      Russ: Even small companies, I just don’t think that should be a policy goal, to have exports for exports’ sake, because I just think it leads to too many problems.

      Noah: Japan does a lot of things wrong, but it also does some things right. And it has an export promotion agency called JETRO. If you’re a small Japanese company, you don’t know how to sell stuff to Bangladesh. JETRO will hook you up with local resources, and will try to hook you up with some financing, and things like that to basically nudge you toward these markets. Yeah, I think America could absolutely use something like that.

      Russ: Well, we have the EXIM Bank, which mainly works as a subsidy to Boeing, not small companies.

      Noah: That’s, yeah, right.

      Russ: And they claim that they’re good for small business. And I think the political — what you would design, Noah, and what I might design if I were in agreement with you, would be different than what we’re going to get.

      Sonal: So we ended on the note that China had this unprecedented scale, this shock effect, because of the fact that it was a one-time tenure, one-fifth of the world. We’re talking about the past. So what happens next when it comes to — we think about the evolution of technology. We’ve alluded to, you know, automation and what’s happening with jobs. But just more specifically what happens next?

      Russ: The easy answer is, we have no idea. But the second answer which is, I think, going to be true no matter what happens is that it’s going to be very hard to do something about it even if we don’t like the outcome. Technology is incredibly powerful right now, innovation is incredibly powerful. We’re going to get autonomous cars in the United States if the technology works out. I don’t think the political power of the taxi cab industry is going to stop it, just like I don’t think the hotels can stop Airbnb. And so I think we should be focusing on the costs of that transition and preparing people…

      Sonal: Education, etc.

      Russ: …with education and better skills to cope with it.

      Noah: I couldn’t say that any better. Russ, I completely agree.

      Russ: You guys finally agree completely with each other on this one.

      Noah: I mean, on this point, you can’t stop technology. And you can’t suppress it. These things are coming. CRISPR is coming to design your babies someday.

      Russ: I was going to say, yeah.

      Noah: Driverless cars are coming. We’re going to have to roll with it.

      Sonal: Yeah.

      Noah: We’re going to have to be flexible. We’re going to need things like continuing education throughout people’s entire life, and not just, you know, teach them stuff and then release them into the workforce. We’re going to need all kinds of things to become more flexible and to roll with these things. Because trade in the 2000s was something we might’ve been able to manage better than we did. But that’s over, and the challenges of the future are completely different than the challenges of the past.

      Russ: Well said.

      Sonal: Thank you for joining the “a16z Podcast,” guys.

      Russ: Thank you.

      Noah: Thank you.

      • Russ Roberts

      • Noah Smith

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Curious Case of the OpenTable IPO

      J.D. Moriarty, Jeff Jordan, and Sonal Chokshi

      There are the things that you carefully plan when it comes to an IPO — the who (the bankers, the desired institutional investors); the what (the pricing, the allocations); and the when (are we ready? is this a good public business?). But then there are the things that you don’t plan: like the worst financial crisis since the Great Depression… as happened before the OpenTable IPO. There’s even a case study about it.

      And so in this episode of the a16z Podcast, we delve into those lessons learned and go behind the scenes with the then-CEO of the company — now general partner Jeff Jordan — and with the then-banker on the deal, J.D. Moriarty (formerly head Managing Director and Head of Equity Capital Markets at Bank of America Merrill Lynch), in conversation with Sonal Chokshi. Is there really such a thing as an ideal timing window?

      Beyond the transactional aspects of the IPO, which relationships matter and why? And then how does the art and science of pricing (from the allocations to the “pop”) play here, especially when it comes to taking a long-term view for the company? What are the subtle, non-obvious things entrepreneurs can do — from building a “soft track record” of results to providing the right “guidance” (or rather, communication if not guidance per se) to the market? And finally, who at the company should be involved… and how much should the rest of the company know/ be involved? In many ways, observes Jordan — who got swine flu while on the road to the OpenTable IPO — “your life is not your own” when you’re on the road, literally. But knowing much of this can help smooth the way.

      Show Notes

      • The history of the OpenTable IPO [0:47]
      • A discussion of pricing [10:09] and details around the IPO process behind the scenes [16:20], including mistakes along the way [21:02]
      • Key takeaways from the IPO experience and advice for others [24:00]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today we’re doing one of our war stories podcasts, where we have founders, makers, and operators share the story behind the story. And, joining us for this episode, we have a16z general partner, Jeff Jordan, who was president of PayPal at eBay before going to online restaurant reservation network, OpenTable, where, as the CEO, he oversaw the company going public.

      We’re gonna talk about all that in this episode, focusing on everything from the relationship-building involved on the road to IPO, and the nuances of pricing and allocations, to the broader market context, and some concrete advice for entrepreneurs. And, last but not least, we have special guest J.D. Moriarty joining this conversation. He’s now SVP of corp dev at LendingTree, but was formerly managing director and head of equity capital markets at Bank of America, Merrill Lynch.

      Background on the OpenTable IPO

      Jeff: J.D. was the lead banker, the capital markets expert from Merrill Lynch on the OpenTable IPO. He and Harry Wagner of Allen & Company were the key — two keys who basically helped execute a deal in about the worst capital market situation in late…

      Sonal: When was that?

      Jeff: Late 2008, early 2009. A venture-backed technology firm had not gone public in a couple of years. The concern was that the window was closed, bricked-over, and the only exit path for tech companies was going to be M&A from then on. We ended up pricing at the nadir of the worst financial crisis since the Great Depression.

      J.D.: I used to talk about 200 IPOs a year, and the only IPO prior to OpenTable in 2009 was a company called Mead Johnson, so a very defensive company, that type of thing that should go out in 2009 in consumer products.

      Jeff.: They made, like, the floor wax or something right?

      J.D: Yeah, exactly.

      Sonal: Very far from this weird thing called OpenTable, which is not even a product you can physically touch.

      J.D.: Correct. And so, you know, people tend to look at when an IPO prices — and you have to recognize that most companies take six to eight months to get there. From our org meeting to kick off the process where the bankers and the management team begin the process of preparation, our pricing was about eight months.

      Sonal: I have to ask a really dumb question. Why do you need a bank? Like, why can’t you just directly IPO, bluntly?

      J.D.: Actually, at one point, Jeff asked me early in the process, “Hey, why do we need to go see all these investors? Can’t we just do a $35 million IPO to four or five folks?” I think the way to think about the IPO process is, you’re not just doing the IPO. You have to take a two-year view towards, how do we get this to be a stable public company that can grow — and achieve not only the company’s goals, but the goals of the early investors, with regard to monetization over a long period of time? And oftentimes, as this deal showed, those goals are different.

      Jeff: It is an interesting observation is — going public does not create liquidity. All the insiders cannot trade when you go public.

      Sonal: Why is that?

      J.D.: Well, a standard expectation [is] that the new public investors — taking a chance on this new company — is a 180-day lock up. That is kind of a market standard.

      Sonal: The lock up period, right.

      J.D.: There are certainly exceptions to that. And we can talk about things like the IPO discount, etc. But in order for somebody to take the risk of a newly public company, they expect certain things. Now, there are plenty of IPOs that have secondary shares, don’t misunderstand. But the early investors are locking up for 180 days. There is a period of time when the right way to think about it is, your true monetization is really down the road.

      Jeff: Yeah. And so, if when the 180-day lockup expires, all the insiders and all the management team run to the floor of the stock exchange and try to sell all their stock, all the third-party — all the owners will disappear too. Because if the insiders don’t have any confidence in it, then why should I own it?

      Sonal: I mean, just to give a little bit of a counterpoint, though. Obviously, market conditions change. If the company hasn’t gone public in, like, 10 years, and you have to get some liquidity out — or you’re a founder who has to give up a little bit of shares in the secondary market in order to, you know, loosen up your…

      Jeff: We are in support of that, but if they try to get full liquidity on day 181, the stock price is gonna be, like, $2.

      Sonal: Yeah, it’s the reality of how the market works.

      J.D.: Yeah. We’re talking about the technicals of, how does that stock get to market? There’s another part of this, which is simply, when do you time the IPO? Fundamentally, we encourage people to — don’t think about the market conditions today. Think about, is this a good public business? And you have to answer that question first. And for most companies, you’re never gonna time the market.

      Sonal: You can’t time the market because it’s clearly a long process. But then why do people talk so much about there being certain windows, in which there is an ideal time that, you know — like, a window can open and shut, and this is on a global scale of 10 years, 15 years, etc.

      J.D.: This was actually — there’s a case on the OpenTable IPO. It’s at Stanford Business School now, taught in the “Formation of New Ventures” class. Andy Rachleff wrote it. And it basically says, should OpenTable go public now? The analysis said, there are windows in the IPOs. Yeah, they kind of come and go, and the best companies often open windows that were then previously closed.

      Sonal: Interesting.

      J.D.: So, the best companies can go out whenever they want. The good companies typically wanna wait for — till the investors are feeling good. The mediocre and bad companies wanna get out whenever they can. And what happens — when the window opens, the early people to go out — typically perform very well as public company stocks. And then, as more time goes by, and you get towards the end of a window, the companies that then are going out — kind of rushing, “I gotta get out or I’m going to miss it” — are typically not the highest quality companies, and they tend to underperform the market. One of the things that had us go was, okay, we think we’re a good company and we think we can perform in the market. We wanted to meet bankers ahead of time.

      Sonal: And how far in advance did you do that?

      J.D.: We started about a year and a half out or something like that.

      Sonal: A year and a half?

      J.D.: Just surgically, just with the bankers. We kind of orchestrated into one or two conversations. But before we did the formal bake off, where you select your lead bank — and all due respect, Merrill at the time, they were, you know — they were not the top of the pyramid, in terms of tech bankers. So we — because we were the only IPO, we had every bank [that] wanted to do it.

      Sonal: So how do you pick?

      J.D.: Goldman, Morgan, you know, just down the list…

      Sonal: Yes, so how do you pick them?

      J.D.: We got to know the people at the firm. And we put particular value in the capital markets function, because that is the function that interacts between the company and the investors. We wanted someone who we thought understood our business well, and we thought could represent our business well to investors.

      Sonal: And by investors, just for clarity, you mean investors like institutional investors?

      J.D.: These are institutional investors who invest in public securities. Typically, the largest ones are mutual funds, managing billions and, you know, billions of dollars. We were looking for tentpole investors who would go for a while, and we’d meet with them every six months or so. And they got to know the business. They got to know us. They got to — we’d developed a soft track record. Because they’d say at the first meeting, “What are you gonna do in revenue this year?” “Oh, we’re gonna do $70 million.” Then you come back six months later, “What did you do?” And we said, “We did $80 million.”

      Sonal: Oh, you mean, like a soft track record of delivering results. It’s almost like quarterly reportings.

      J.D.: Of performance, yeah.

      Sonal: But they’re, like, informal — verbally.

      J.D.: Yeah. Am I the kind of person who overhypes and under delivers? Or do I under hype and over deliver? Or do I tell them what’s good about the business and what’s bad about the business?

      Sonal: That is such a golden nugget, because it’s invisible to the world. When you see the outcome — the process behind the outcome is invisible, which is the whole reason we’re doing this. I didn’t know that. Why does that relationship that the capital markets expertise matters so much in the lead up to the IPO?

      Jeff: We wanted it to be not a black box. One of the things other CEOs had told me who had done IPOs when I reached out, it’s like — somehow, you know, you do this roadshow, you get to the pricing meeting. They say, okay, we recommend the price is this, and then the shares just magically disappear. And we really cared about who got the shares. So, we wanted to have a vote in who got it, who got the shares.

      And, because it was such a tiny offering, we wanted to concentrate the shares in that shortlist at a much higher level than what’s typical at the pricing meeting. You know, we — J.D. shared the spreadsheet, and we’re like, “No, no, no, we have to give these guys 10 times more.” And he’s going, “No, no, no.”

      Sonal: Wait, so who voted in the — it was like the, you know, the bank, you guys thought…

      J.D.: Well, typically, to Merrill’s credit, typically, the bank pretty much decides. They engaged in a dialogue with us. And that’s probably the biggest thing where we met in the middle, said, “Okay, I understand your rationale.” So, we had this very constructive dialogue around that. In many IPOs, that dialogue does not happen.

      Sonal: That is such an artful, behind-the-scenes orchestration.

      J.D.: It’s all a hangover from the ’99, 2000 period, when those allocations were truly a black box. And somebody said to me at one point, is there any innovation in the IPO market? And I said, the biggest change over the last 10 years is that it’s become more transparent. And that is more the norm today — that there’s genuine conversation around it. Now, I’ve seen the other side of it, which is a management team says, “No, it’s gonna be like this.” Ultimately, their vote is the vote that matters.

      But I’ve seen scenarios where they make mistakes there too. What the OpenTable team did well is actually invest in not just the bankers, but actually the investors. The thought process was, the better they know our business. These are the people who are gonna have skin in the game and really own enough of our stock.

      Sonal: Hold the tent up.

      J.D.: And know where the business can go and hold the tent up in a difficult time. And so, Jeff essentially invested in that process, and I think it was critical. One of the debates we had was size of IPO. 

      Sonal: Like, the amount of the initial public offering.

      Jeff: The proceeds, yeah. And so we spend a lot of time doing analytics for companies on how big does your IPO need to be? What does your market cap need to be? How big do the proceeds need to be? And if you — if you just, sort of, look, it doesn’t pass the common sense test, right? Why does some portfolio manager at Fidelity, who manages billions and billions of dollars, invest in the OpenTable IPO, when the initial proceeds are only gonna be $37 million?

      Now, ultimately, because it went well, we ended up raising just shy of $70 million in the IPO. And then, in September, we ended up doing a follow-on transaction that was $210 million. But the point was, why is it worth it to Fidelity, Morgan Stanley investment managers, T. Rowe Price.

      Sonal: Yeah, so why is that? I wanna know.

      J.D.: Because if you — if you invest the time to let them see where the business can go over time, they’re gonna leg into their position over time. You know, if we had been — we were in a lousy market. One expression we always use is, “In difficult times, our investor clients focus on the things they own, not those things that we wanna show them.”

      Sonal: What do you mean by that?

      J.D.: Meaning, new ideas. So, it’s just a risk curve issue. And Jeff made the point about great companies being able to go out in any market. Good and okay companies need to pay more attention to the investor risk curve.

      The question of pricing

      Sonal: Back to the notion of pricing. We had Lawrence Levy, who is a former CFO of Pixar on this podcast, and he’s the one who helped Steve Jobs take Pixar public. And one of the things that he talked about — how it was the biggest fight between him and Steve. And the reason was because, of course, he wanted to go high — because he wanted a big-ass IPO, like Netscape at the time. And he was on the heels of that. And Lawrence was like, “No, no, you wanna deliver some returns for the investors.” And there was sort of this dance back and forth. How did you guys sort of do that dance?

      Jeff: Yeah. A lot of discussion.

      Sonal: Is that a euphemism for fighting?

      Jeff: No, no. There’s a lot of discussion. You know, our at IPO market cap was $450 million, roughly. Raising proceeds of just under $70 million.

      Sonal: The IPO size was $70 million.

      Jeff: Now, I’ll be the first to admit, did it trade, kind of, too well? Yes.

      Sonal: So why is it bad if it trades too well?

      Jeff: There’s too much of a pop.

      J.D.: It means the company didn’t get as much money as they could have if they had a crystal ball and knew what…

      Sonal: Right, because the whole point is to get capital to continue growing and building the business, I get it.

      J.D.: Correct. You don’t wanna leave a lot of money on the table. Now to be clear, when you end up floating a small amount of the business, you kind of compound this problem.

      Sonal: Why is that?

      J.D.: Go back to the point around the Fidelity’s and T. Rowe’s needing larger position sizes — they recognize that the two events that they care about are the distribution of shares at IPO, the allocations which we went back and forth on, and the first day of trading. And then these stocks become very, very illiquid.

      Sonal: They’re in the public market, how do they become illiquid?

      J.D.: Because the float is only $70 million.

      Jeff: And we convinced people that it was an interesting stock to buy and hold. That meant we had no daily trading volume.

      J.D.: So, if the stock was trading around $30 a share, there were days when it was trading, literally, 2,000 shares. 2,500 shares.

      Sonal: Not many people moving money.

      J.D.: Correct. And so, you can get these huge gaps. So it did trade “too well.” That’s a balance that we’re always trying to strike.

      Sonal: Yeah, so from your perspective, Jeff.

      Jeff: Yeah, I know. So, when we — the original documents had a $12 to $14 price range. I think we updated it to $16 to $18 over the course of the roadshow, because the roadshow was going well. The first two investors said, “I want a full allocation.” So, it quickly became a hot IPO. We ended up being oversubscribed, 20 to 1, 25 to 1, something like that. So, we probably could have run it up into the mid-20s easily.

      Sonal: But you guys priced it at…

      Jeff: We talked to the market to 22, and we priced at 20. And most management teams — board CEOs are gonna grasp for that last dollar. I think the OpenTable team collectively was very thoughtful about the fact that, this is just the IPO. I care about the next two years.

      Sonal: Did you guys — tell me the truth. Did you guys have, like, a magic number in your head before you start those pricing discussions? Like, did you think in your head, you know what, when I go to sleep at night, I want $25 when this thing goes on the market?

      J.D.: No, we didn’t, you didn’t. Part of our strategy was, we’re gonna do a teeny, little IPO. And then if it went well, we were gonna do a pretty big secondary. And so, the company was much more focused on “make the secondary successful” than it was “make the IPO successful.” Part of making the secondary offering successful was, you needed a couple of deep pocket people in the IPO, even though it was a teeny, little IPO. So, one of our leading shareholders ended up being whittled down off of Fidelity.

      And so, when we say, “Will, invest out of your $10 billion — whatever it is — fund, $4 million. And he’s like, ‘I don’t have the time to read your earnings release at that level.’” But we convinced him to come in, because then, in the secondary, he was able to back up the truck, and he got what he wanted, which was a larger ownership allocation. His IPO allocation was, what — 5% of 70, so $4 million, some number like that.

      Jeff: Yeah, and one of the things that made the add-on so much easier is because everybody knew that we could have priced well above $20. When we priced at $20 that, I think, built some real goodwill between the management team and the investors. 

      Sonal: That you guys are willing to be thoughtful about the long term.

      J.D.: And we ended up optimizing for who got the shares, not the price they got the shares at. And I would do that again in a second. I’d advise management teams to do it like crazy, because as a CEO managing a public company, you don’t want people who are in and out on momentum, hot money — because you’re spending, then, all your time literally marketing to new investors, “Please buy my shares.”

      There are multiple reasons to want a small — from my perspective, to want a small handful of tentpole investors who buy and hold your stock. One is, they buy and hold. The other is, it just makes your life easier. I mean just…

      Sonal: Because you’re only, like, talking to a pool of four to five people?

      J.D.: Yeah, I’ve got a handful of owners, and most of them — I actually said, “How do you want me to work with you? You own a lot of my shares. Do you want me to call you after every earnings call?” And they’re like, “No. I’ll listen to the call.” Almost all of them were just like, “Nope, I’ll reach out if I need anything. Thank you very much.”

      Jeff: In the first year and a half, there was a period where we dealt with the momentum crowd coming into the stock.

      J.D.: Late.

      Jeff.: And it was challenging.

      J.D: That sucked.

      Jeff.: Because we essentially went from, you know, the projected — keep in mind, when we’d gone public, in 2009, the projected top line growth in the business was 20%, from an analyst perspective, just under 16% to 20% depending on what the analyst — it was always a high margin business. And then it was when you went to 40% top line and 40% margin, that’s when every momentum investor came in. And so, that was one of the things that I think became more challenging to manage.

      Sonal: And momentum investors, you mean, like hedge fund people?

      Jeff: Not just hedge funds, but in many cases, it is. That’s a broad stroke. Ultimately, they’re investors who just care that you’re gonna beat the quarter. 

      Sonal: Yeah.

      J.D.: I mean, it was so interesting, because there were a handful of investors, before the IPO, who were tracking the business. So, I was told that Fidelity — Will does not do IPO pitch meetings. Will walks into the meeting and spent the whole 60 minutes there. Dennis Lynch at Morgan Stanley does not do IPO meetings. Dennis was early. He was waiting for us when we got there. You’ve got those who you’re like, “Okay, they’ve done their homework, they get it, network effects, everything else.” Dennis can tell you what three private companies he wants an IPO allocation in today, for five years from now.

      Sonal: Because they have a strategy, they think about it.

      Jeff: They have a strategy. The other guys, you’ve just blown away six quarters and said, “Oh, my god, I need to latch on to that sucker.”

      Sonal: Yeah, get me into that.

      Jeff: It’s like, “Where were you when it was $20? You’re buying at $110 now.” They’re the guy that will jump in, but they also jump out. That’s when stocks freefall.

      Behind-the-scenes IPO process

      Sonal: Tell me about any behind-the-scenes fights or discussions that you’ve had with your team. Like, were there disagreements? I mean, you might — you’re saying this, but were there parts where you guys were like, “We can’t agree on the pricing that these guys are discussing with us. We can’t agree on the timing.”

      J.D.: Not a ton. The board gets involved at a few steps along the way. One is, I involve them in the selection of bankers. They were there.

      Sonal: Is that a best practice, that you advise people should actually involve their board in that?

      J.D.: I did. We did a bake-off. We had, like, six people, and we tried to do “wisdom of the crowd.” No one was allowed to say who they liked and didn’t like, and we got a ranking one to six. And it was unanimous. We did a subgroup that got — went deeper than the rest of the board, the IPO committee. Then the board also gets involved in things like, okay — do you launch the roadshow? And what’s the price? Almost all of the decisions — the process is being run by the management team. And so, in our case, it was CFO, Matt Roberts, and myself. And we insulated the entire rest of the company from it.

      Sonal: Wait, so you did not involve the rest of the company? It’s just you guys?

      J.D.: No, they’re out there back in San Francisco building the business.

      Sonal: Building product.

      J.D.: We’re spending two and a half weeks running around the country.

      Sonal: Is that typical? Is it the CEO and the CFO that you typically want in the room?

      Jeff: Yeah. And in fact, one of the mistakes that we see companies make periodically is one, building employee expectations towards an IPO too early. And two, involving too many people. If you think about it, one of the things that larger companies, private equity-backed companies tend to do well, is they value the option value, they get themselves ready to go. But guess what? They’ve also got more resources at the company to do that.

      Sonal: Wait, can you break down what you mean by option value? That’s a very loaded — those are two very loaded phrases in our business.

      Jeff: Sorry, the preparation, the preparation phase. And with venture-backed companies, I think you have to be mindful — and the boards are mindful of the fact — of the distraction that you can create through the IPO process. And so, what Jeff and Matt did was say, “You know what, this is going to be borne by us. Everybody else, do their job.” Everybody else was focused on doing their job and managing the business. Go back to when we were talking about the odds of it being a lousy market, that we might say, “You know what, this is not our year to go.” We’re there. So, it would have been a huge distraction.

      Sonal: One of the things we haven’t talked about is that, you know, in a lot of cases, the IPOs involve novel technologies that are not familiar to the market, and you’re essentially selling a new way of doing things. Now, it’s very familiar to us, to actually book our reservations online. But how do you think about involving other key, like, technical people to help sort of educate, or is that the CEO or the CFO? Don’t you feel frustrated as a founder, that my CFO can’t represent this that well?

      J.D.: I had a very good CFO. Matt Roberts did a fantastic job. He actually was the one who orchestrated almost all the IPO till the roadshow. But if your CFO can’t tell the story, you need a new CFO.

      Sonal: Well, that’s why I’m asking, you’re honestly, when I think of a CFO — no offense to all the CFOs out there — I think of number people who are just sitting there with, like, spreadsheets.

      J.D.: Different CFOs can have different styles. The investor has to trust them. And then that’s what the investor is looking at. “Is the CFO telling me the truth, know what’s going on in the business?” And how does that play?

      Sonal: Is there one thing that you’re like — I want every CFO to have this, from both of your perspectives?

      J.D.: Integrity, attention to detail.

      Sonal: That too.

      Jeff: That’s true.

      Sonal: The market wins that too. My mom with her, like, 15 Snapchat shares wants that too. I mean, we all want that.

      Jeff: We often ran into management teams that wanted to have too many people on the road. Three people is, sort of, the outer number, and…

      Sonal: Who is the third, by the way, when it is not the CFO and the CEO?

      Jeff: To your point about — it depends on the business. To your point around technology. Periodically, if it’s a highly technical business, you might suggest you have that person there for Q&A, but you don’t want to have the distraction. You wanna have dialogue. Most investors expect CEO, CFO, dialogue. So when you get to the CFO question, we can tell which teams are gonna need a lot of preparation, and it’s seldom both CEO and CFO. And then we just — we hit him with questions. These are the types of questions you’re gonna…

      J.D.: It was the CEO, in my case. He just doesn’t wanna say it loud here. You’re in a different city every night. You’re doing, like, seven meetings a day, hour-long meetings a day. And the thing they stress more than anything is — give exactly the same presentation, and answer every question exactly the same, because of regulation, FD, fair disclosure. Literally, they said no, no, no, don’t be playing around with giving that slide two different ways. You give that slide one way.

      Sonal: You’re like a robot.

      Jeff: We did it 42 times in, like, two weeks.

      Sonal: Oh my god.

      J.D.: All in different cities.

      Jeff: You get really tired of hearing your own voice.

      Sonal: Yeah, I can imagine.

      Jeff: And if you don’t, I’m not sure I wanna work with you.

      J.D.: No, it’s unbelievable. One really helpful thing is, the OpenTable core customer included bankers living in New York, Boston. Earlier in my career at eBay, none of those owners would use eBay.

      Sonal: Interesting.

      J.D.: Unless they happen to be collecting something because, you know, time is much more valuable to me than money, and e-Bay was a cost-saving thing.

      Sonal: It strikes me that that’s actually one of the challenges, because OpenTable, then, is a consumer business. It’s one of the challenges of enterprise-facing businesses, and especially SaaS businesses, where the financial model is also not as familiar for people to talk about. 

      So, I wanna talk about the bumps in the road now, and the unexpected things that happen. I mean, there’s a lot you’re doing in the road up to the IPO — a lot of prep, clearly. But despite all your hard work, unexpected shit happens.

      J.D.: It happens in every one. I wasn’t at eBay at the time, but I believe Amazon launched their auction competitor on — during either the IPO in the secondary, and Yahoo launched their auction competitor on the other one.

      Sonal: Oh man.

      J.D.: We had two big ones. One is, we got our obligatory patent troll lawsuit. That happened while we were on the road. They wait until you’re on the road, point of maximum leverage.

      Sonal: Of course, they do.

      J.D.: And file.

      Sonal: They can get their money, get you out of the way.

      J.D.: So, I get the call, you probably, you are saying, “Oh, by the way, you just got served.” You’re like, no. So that was one. The other one was a little more self-inflicted. And it turned out, no one had done an IPO for a long time, including the SCC, our accountants, and our attorneys. And our attorneys, at the last minute, update the filing, you know, they keep — it’s very formal. They update the filing the night before, and the SCC gets it, and they say this is approved, and you’re ready to sell the next morning. So, after a bottle of wine that night, when you have the price.

      Sonal: And you’re, like, relaxing.

      J.D.: We’re trading the next morning and I wake up a little early for — like, 3:00 a.m. Yeah, I mean, you’re just wired. Go to the gym. I’m working out, I open up my smartphone and see …

      Sonal: Was it a Blackberry at the time?

      J.D.: It probably was a Blackberry at the time, because this was 2009. “Offering on hold.”

      Sonal: Why? What happened?

      J.D.: Turned out the attorneys had — when they did the last turn, had attached the wrong attachments. The SCC approved something…

      Sonal: Human error.

      J.D.: …that we actually knew was erroneous. And so, if we started trading on an erroneous document, there’s a chance the SCC can say, “No, no, no, you have to unwind all those trades. Go start over.” Now, you’re just like, it’s in every newspaper. The business section in America. We’re going public today. Now we’re not going public.

      Sonal: Well, what did you guys do?

      J.D.: Our attorneys sort of start — all the attorneys on the deal, just start trying to get the SCC on the phone. And so, we ended up delaying the opening, finally right as the market opened the SCC, “Oh no, you’re blessed again.” And so, then started trading an hour or two later.

      Jeff: But we did have a delayed open. We had a significantly delayed open. Not unlike Facebook’s delayed open, just a different outcome.

      Sonal: So just, like, 10:00 a.m. instead of 7:00 a.m. kind of a thing.

      Jeff: Yeah. If you’re on the New York Stock Exchange, you’ll typically open closer to the 9:30 start, and on NASDAQ, they have a — they always have somewhat delayed windows. We were very delayed.

      J.D.: We were very delayed.

      Sonal: And that is time — a case where time is literally money. It’s, like, ticking away if you’re not…

      J.D.: Yeah. Now you know I’m obsessed with Hamilton.

      Sonal: We both are.

      J.D.: One song says, he walks the length of the city. They had me show up about a half-hour after trading has started, so I don’t walk into a potential disaster. That’s, like, four or five miles, just through the city. It starts trading well. He goes, you’re in good shape. You can go home. And so, I walked back.

      Sonal: Wow, you know what I mean.

      J.D.: And then I walked across. And then I was just like, oh my god.

      Sonal: What about that whole “ring the bell” thing? Didn’t you guys wanna, like, be there when they’re doing that whole thing?

      J.D.: We didn’t end up having it the same — we had delayed.

      Jeff: Which, by the way, is the weirdest thing, because it’s a soundstage on Times Square.

      Key takeaways and advice

      Sonal: That’s amazing. Okay, wrap up and takeaways, relationships matter, timing matters. But, while I understand your earlier point, that you can’t time the market itself, the context — the broader environment — does matter. And how do you, sort of, look at back then — that was 2009, and now, 2017? It’s eight years later? What are some of your views on how IPOs have changed given this context?

      J.D.: So, a couple of the things that are big takeaways from the open team have actually been somewhat formalized in the market. And so, one of my big takeaways from this transaction was, what the team did well was invest in the process, be ready to go. They valued the option value of being ready. They invested in that, and then were able to respond to an open window, essentially. The other thing that Jeff highlighted was spending time with the public investors, long before that 45 minutes to an hour-long meeting when you’re on the road.

      Well, post the JOBS Act, that second part has been somewhat formalized. You can do that more easily today. It’s called “testing the waters” meetings. Now, some management teams probably place too much value on it. To your point, you weren’t going and meeting with a cast of thousands. You were meeting with a narrow group of believers.

      Jeff.: A half dozen. A half dozen.

      J.D.: All right. And so, what I tell people is, beyond a certain number you hit diminishing returns. It is particularly helpful for a unique business that you don’t think you can get a full appreciation for in that one-hour meeting. And so, it’s a business-to-business discussion, but have a discussion with your bankers around whether the “testing the waters” meetings have value on a relative scale for you. But that’s something that the market has enabled with the JOBS Act. Layer that into your timing equation.

      I think the other piece of advice I’ve given people is, time the IPO for your business and your team — your team, including your board and investors. Don’t time it around the market. Time it around the right time for your business. Think about the scale that you need to be at to be a public company. One of the questions we get is, you know, what market cap is too small? Well, below certain thresholds, we just shrink the number of investors who will buy the deal. And that’s not a good leverage thing for the company.

      Sonal: But there are small-cap IPOs out there that are okay?

      J.D.: There certainly are.

      Sonal: In fact, that seems to be a growing trend in some ways.

      J.D.: Yeah, there certainly are. I think that you have to think about — how unique is the business? If there are four public companies that give an investor the same exposure, and three of them are of decent market cap, and you’re going to be the very small one, you better offer something different. OpenTable was certainly unique, and thus, way more leeway from the market.

      Jeff: There’s also, I mean, you can go public too early. You can go public too late. And if you want a multiple, you wanna be a growth stock.

      Sonal: Why does that matter to be a growth stock?

      Jeff: You get a different fundamental valuation and a different set of investors who are willing. If your growing investors can say, “Boy, if they keep that up for five years, look at that envisioned future. That’s wonderful.” You’re not growing, they look at it and say, like, “It ain’t gonna get any better than this.”

      Sonal: And it does seem like the best technology companies are like that.

      Jeff: Oh, yeah, it’s — you want to have the perception you’re a growth company, which actually means you have to be delivering growth results. And typically, growth rates decline over time. We were in the teens — year over year growth rate, according to the analysts’ models — when we went out. That’s not really a strong growth company. Then we increased growth in the 30%, 40%. That was a growth company. So if you wait too long — but too early, if you’re not ready to be a public company, you can — you know, your business is unpredictable, you know, a bunch of things there. So there is this element of, okay, the biggest timing is from the company’s perspective, not from the market’s perspective.

      Sonal: That’s a really great shift in mindset. So, just one quick thing then. You mentioned results. And that is obviously, like, the quarterly results, the earnings calls, and the whole song and dance that goes around that. How do you manage that?

      Jeff: So, well, two thoughts, one is on the timing of going out. When you go out, you don’t want to miss the first x quarters — [don’t know] how many x is — but you wanna be highly confident you can exceed the expectations that your investors have. So, I usually counsel CEOs, before they go out, have something in your back pocket. We had two or three initiatives that we tested at small scale, that we knew were gonna work and add to the business. So, I was highly confident.

      Sonal: But isn’t there a tension, though, that — of course, you wanna make your numbers and I get that’s important for the market confidence in you as a company. But it’s also very frustrating, because — a criticism people say about IPOs is that, then, you’re now wedded to this ridiculous quarterly measurement of innovation.

      Jeff: I think that’s whether you will allow yourself to be or not. The pressure is there to conform to investor expectations. You know, “Oh, my god, I’m gonna miss their number.” That’s the most oft-cited reason for not wanting to be a public company. Look at Jeff Bezos. He’s run the business exactly as he wanted. He told investors exactly how he’s gonna run it. He’s been completely consistent with that. And some of his investors bought in his IPO. They’ve stayed with him, what? 20 years now. And so…

      Sonal: I actually heard some statistic just yesterday, because of its anniversary, that some of the people who got the early IPO, they might have only spent, like, $100, and it’s now worth $64,000.

      Jeff: So, part of it is, what investors did you recruit? And then how do you communicate with them? So, we had an interesting early question. Do we give guidance? Do you just say, “Next quarter, we think we’re gonna do revenue of x and earnings of y.”

      Sonal: Isn’t that what analysts do?

      Jeff: Well, analysts do that. But often the company gives a range, and that helps the analysts cue in on the range. If the company doesn’t give it, you’re leaving the analysts to come up with their own numbers. I called — tough question — [should] I give guidance. I called the three largest holders, and I said, “Should I give guidance?” All three said no. I go, “Why?”

      Sonal: Why?

      Jeff: And they go, “Well, we want you to do what’s right in the strategic long-term interests of the business. If you give guidance, there’s gonna be pressure for you not to do what might be right, with new learning.” And it helped that our business was highly predictable from outside. You know, it’s not one of these, like, did we close the last deal on the last day of the quarter? You know, the diners are being seated by the millions. And so, the law of large numbers kicked in. It was very predictable.

      J.D.: With great metrics. Like, you guys invested in explaining the metrics that matter to you as a management team. Periodically, we hear people get this mantra of no guidance, and they interpret it as no communication. They’re two very different things. Right? You were not signing up to a quarterly number. But it was fine because you were giving some much transparency as to what drives the business.

      Jeff: Right, they can build their model.

      J.D.: They can build their model.

      Sonal: Okay, so any last parting thoughts?

      Jeff: Taking OpenTable public was one of the most interesting things I have done in my business career. And part of it was I — turns out, in my career, I’d never done a financing. So, for my first financing to be taking OpenTable public, in May 2009, at the depth of the financial crisis.

      Sonal: The worst time.

      J.D.: It was an amazing learning experience, and there’s a lot of emotion into it. I think I was more exhausted — I ended up with the swine flu at the end of the process.

      Sonal: Swine flu. I haven’t heard that phrase in a while. That totally ages that time again.

      Jeff: It was the height of the craze on that too. Not only is the whole economy coming to an end, but we’re all gonna die. And so we knew it was a problem. We’re walking around with bottles of Purell. But we shook 400 hands. Your life is not your own.

      Sonal: Thank you for joining us in this podcast.

      Jeff: Thank you. Thank you, J.D.

      J.D.: Thanks, Jeff.

      Jeff: It’s good to see you.

      • J.D. Moriarty

      • Jeff Jordan is a managing partner at Andreessen Horowitz. He was previously CEO and then executive chairman of OpenTable.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Making a (Really) Wild Geo-Engineering Idea Real

      Ross Andersen, Hanne Winarsky, and Sonal Chokshi

      Here’s what we know: There’s a pair (father and son) of Russian scientists trying to resurrect (or rather, “rewild”) an Ice Age (aka Pleistocene era) biome (grassland) complete with (gene edited, lab-grown) woolly mammoths (derived from elephants). In Arctic Siberia (though, not at the one station there that Amazon Prime delivers to!).

      Here’s what we don’t know: How many genes will it take? (with science doing the “sculpting” and nature doing the “polishing”)? How many doctors will it take to make? (that is, grow these 200-pound babies in an artificial womb)? What happens if these animals break? (given how social elephants are)? And so on…

      In this episode of the a16z Podcast — recorded as part of our podcast on the road in Washington, D.C. — we (Sonal Chokshi and Hanne Winarsky) discuss all this and more with Ross Andersen, senior editor at The Atlantic who wrote “Welcome to Pleistocene Park“, a story that seems so improbably wild yet is so improbably true. And while we focus on the particulars of what it takes to make this seemingly Jurassic Park-like story true, this episode is more generally about what motivates seemingly crazy ideas — moving them from the lab to the field (quite literally in this case!) — often with the help of a little marketing, a big vision, and some narrative. And: time. Sometimes, a really, really, really long time…

      Show Notes

      • The problem of melting permafrost and climate change [1:06]
      • Why scientists want to create a modern woolly mammoth [7:45], and the gene technology they may use to do it [15:09]
      • The importance of grass in the development of early humans [17:51]
      • How researchers market this project and others like it [25:02]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today, Hanne and I are doing another one of our on-the-road shows from Washington, D.C. And today’s guest is Ross Andersen, senior editor for The Atlantic’s science, health, and technology coverage. And he wrote a story earlier this year, in the April issue, called “Welcome to Pleistocene Park,” which you don’t have to have read to follow this conversation. But here’s what you do need to know. A small group — a very small group, in fact — of Russian scientists in Arctic Siberia are trying to resurrect an Ice Age biome, complete with lab-grown woolly mammoths, through a scheme for rewilding grassland instead of forest. 

      And while we focus on the particulars of all that, in this episode — in a hallway style riff beginning with the connection to climate change, and then moving to gene editing, to discussing the science of paleontology, and the sociocultural and economic aspects of radical geoengineering — this episode is really more broadly about what motivates seemingly crazy ideas, moving them from the lab to the field — quite literally, in this case — through marketing and narrative, which is where we end and begin the conversation.

      The problem of melting permafrost

      Ross: So, when I landed on the website and I see that these guys are trying to rewild all or a great part of Northern Siberia, and Alaska, and the Canadian Yukon with this Ice Age grassland biome — and that they wanna put woolly mammoths there — you know, I had the same reaction that everyone listening to this has, right, which is, like, “What?”

      Hanne: “Jurassic Park. Is it real?”

      Sonal: Yeah, “Jurassic Park,” totally. The Ice Age.

      Hanne: “Is this a joke?”

      Ross: “Who are these crazy people?” Yeah, yeah, yeah. Totally. Yet, I was excited to write the piece. And then the other thing about this project that was really compelling is that it’s not that these guys were only just romantic about bringing the Ice Age back to this huge stretch of the Earth. Their primary motivation for doing it is to act as a climate change mitigation strategy, which is to say that the Arctic is warming very fast, and under the surface in the Arctic is what’s called the permafrost. That’s ice that has been there for, in some cases, tens of thousands of years.

      Sonal: And, in fact, very deep. I read in your article, like, up to, like, a mile deep in some places.

      Ross: Yeah. That part of the world was so rich in grass and in large animals at that time. It’s got lots of, sort of, organic matter, which has lots of carbon in it, in fact, more than, like, the entire output of the United States right now.

      Sonal: Let’s take a step back for a minute. First of all, what’s the connection between the permafrost and climate change? Like, how can a grassland steppe with some fluffy, furry animals stop climate change, bluntly?

      Ross: Totally. Okay. So, most of that part of the world up in the Arctic is covered with tundra. You might think of it as the Arctic desert. Like, very little grows on it. It’s kind of, like, scrub. And what’s neat about grasslands is they actually keep the earth underneath them colder. First of all, they reflect away more sunlight than the darker, kind of, tree regions. You’re already hedging against the warming, right, by having grasslands out there. And in the winter…

      Sonal: Shade.

      Ross: Shade, boom.

      Sonal: They like wearing white on a hot day.

      Ross: And then in the winter, you have — the snow cover is, like, on the grass, is really thin, such that, like, the Arctic cold in the winter when it’s really dark, and it’s just the auroras up there, can really penetrate the ground deep, and keep the permafrost even more frozen.

      Sonal: Well, you actually use the language that it’s, like, locked in some thermodynamic vault…

      Ross: I did. I didn’t want to roll that out.

      Sonal: …which I think is, like, the best way of, “I’m rolling it out for you.”

      Ross: Yeah, thank you. Thank you, thank you.

      Sonal: That’s such a really good way of describing it. And so, what happens when those — because isn’t that a good thing to have all that organic matter? I mean, that creates oil, it creates, you know, this rich ecosystem that fertilizes our grass. I mean, what’s wrong with that melting?

      Ross: What’s wrong with that melting is that bacteria will get at it, and through the process, they will decompose it and release carbon as part of that process.

      Hanne: And it’s melting, not just because of the warming, but isn’t there an ecological contribution to the grass going away?

      Ross: What’s so important about the animals being there is that the animals help to maintain that grassland ecosystem. And the woolly mammoth is involved because woolly mammoths, like many of their elephant cousins, are really good at knocking down trees. In fact, they were excited about it. Like, it was like one of their favorite things to do.

      Sonal: But we could just, like, knock down trees ourselves. Like, why do we need the animals to do this? Why don’t we just raze a shit ton of forest trees, you know, pine trees, whatever, and just create grassland? Why do we need these woolly mammoths to be there?

      Ross: In the absence of mammoths, they’ve just had, like, a huge Russian military transporter out on the plains that they’re literally just, like, slamming into trees with to knock them down.

      Hanne: They’re weeding with their, like, military vehicles.

      Ross: As you’d imagine, throwing out, like, a fleet of tractors that can knock down the trees of the taiga and, like, the entire Arctic region, would be a pretty carbon-intensive activity.

      Sonal: So it’s, like, actually, making the problem worse and trying to solve it versus…

      Ross: Like, we need all the world’s oil.

      Hanne: But wait, can I back up and ask a question? Like, what I was trying to get — why do trees grow up that now are a problem? You know what I mean? That we need to, like, if you — why is the problem starting?

      Ross: Well, one theory is that trees took over. First of all, you had the end of the Ice Age, which created a whole bunch of warming, right? And so the trees, kind of, that helped them spring up out there, but also, in the absence of large herbivores, like the woolly mammoth, it’s easier for trees to, like, spring up. And so, lots of people think that, when these animals went extinct — and we can talk about how they went extinct and some of the really interesting debates around that — that paved the way for these forests.

      Sonal: Actually, one of the things that struck me and I feel like I reference “Sapiens” a lot on this podcast — the thing that just blew my mind is, Yuval Harari paints this picture of how humans are basically the worst predators in Earth’s history, and we’re so tiny relative to this huge megafauna, both on land and in water — from, like, huge woolly mammoths to whales in the ocean — and that everywhere humans move, you can immediately see a decline drastically in the number of large mammals that would walk the Earth.

      Hanne: Yeah. It was so interesting when you talk about this birth period, and also, like, in quick succession, right, just ravaging…

      Ross: That’s the word, yeah.

      Sonal: It is. It is the right word.

      Hanne: …absolutely, like, the wildlife and, you know.

      Ross: Yeah. Yeah, it’s really interesting. A lot of that science has crystallized as our timelines for where humans have shown up in the world have gotten more refined. So, from very early on in paleontology, the consensus was, everyone noticed these large animals had died out at the end of the Ice Age, and they thought, “Well, the end of the Ice Age was just a period of warming, and these animals didn’t adapt.” And then as time went on, it’s like, well, glaciations — like, the Ice Age, was not 3 million years of glacial cold. It was, like, 10,000-year bursts of glacial cold and then interglacials, as they’re called, where things would warm again. And these animals had weathered, like, 30 of those.

      Hanne: These tsunamis. You called them, like, ice tsunamis. Yeah.

      Ross: Yes. And they’d been fine coming out of the other side of them. So why this one, did all of these megafaunas die? Humans show up, everything dies.

      Hanne: Well, not everything. A specific kind of thing, right?

      Ross: Yes.

      Sonal: Grassland played a big role, because you no longer had this advantage where big animals could hide behind trees, or rocks, or big things. And so, humans had to adapt by becoming very good at hunting, like, shooting with spears or fire, in order to attack these animals and essentially learn coordination as they got out of trees.

      Ross: Well, one interesting question around that, that I didn’t get to in the early 14,000 word draft — there was more on this — is that it’s always a mystery why Africa has kept a lot of its megafauna.

      Sonal: Why? Why is that?

      Ross: Yeah. So, one of the running hypotheses is that the megafauna of other continents were what’s called naive prey, because, like, humans show up, harmless little thing — whereas, in Africa, the megafauna there had grown up alongside us evolutionarily.

      Sonal: Right, co-evolved sort of.

      Ross: They saw, like, “Oh, these guys appear to be quite dangerous.”

      Creating a modern woolly mammoth

      Sonal: Yeah. So, back to your piece in The Atlantic — reading it, your use of this form of narrative journalism that gets you attached to the characters, the human characters — and I was actually more fascinated by the scientific characters. And that is, the grass, the mammoths, the role of, you know, elephants. And so let’s break each of those down and talk about, you know, what they are and how they connect to this.

      Ross: Oh, interesting. That’s interesting. I never even thought about it that way. I mean, I obviously thought about the human characters. Sergey and Nikita, it’s these two guys, you know, this father and son in the Siberian Arctic, in the very far east, and they’re trying to rewild that part of the world into an Ice Age grassland with extinct woolly mammoths to fight climate change.

      Sonal: Okay. So let’s break down the first character that I think is the most obvious and important one — is this idea of manufacturing mammoths, and specifically the woolly mammoth. Talk to us about that.

      Ross: First of all, one of the other things that really attracted me to this story was — the woolly mammoth, when you talk about animals that are no longer with us, short of the dinosaurs, the woolly mammoth is the most romantic one, right?

      Hanne: That’s so tied to this idea of, like, the first man, kind of. Like, it’s, like, how we have this idea of codependence on this, you know, animal from a very early age, even in popular culture.

      Ross: We can tell, yeah, that’s right.

      Sonal: Even if you think of things like “Clan of the Cave Bear.”

      Ross: Yes, exactly.

      Hanne: Yeah, exactly.

      Sonal: That series by Jean Auel.

      Ross: Although that’s, like, a huge Ice Age mythology, right, like “The Clan of the Cave Bear,” exactly. Yeah, they show up in cave paintings, right? They’re so resonant with, like, this kind of emergence of humans.

      Sonal: And the woolly mammoth, just to give us a visual picture, basically is a big fat Snuffleupagus with tusks.

      Ross: You got it. It’s a furry elephant. And that’s actually quite central to this piece, because if you do want to manufacture woolly mammoths, which is a crazy phrase, you want to do it the same way nature did, which is, you know, elephants were in Asia, in the temperate parts of Asia, before they were up north in the Arctic. As they slowly moved, nature modified their genomes through natural selection so that they had longer fur, and smaller ears, and you know, an extra layer of fat so they could stay warm in the Arctic. It’s nothing more complicated than that.

      Sonal: Except, in this case, it’s happening through CRISPR, and scientists are manually modifying the genes to essentially edit in these characteristics from elephants, which are in the same family.

      Ross: That’s right, yeah. They want to take, you know, basically, an Asian elephant genome and just make really a small number of tweaks. The guy who is really at the forefront of this is George Church, who is a geneticist at Harvard, and kind of has his hands on any number of, sort of, eccentric schemes like this. But I mean, when I first heard about this, I thought, you know, “Really?” But then I started talking to people in the field, and they were like, “Look, he’s out there.” Not “he’s out there” like he’s crazy. George is really at the forefront of this. I mean, like, he has the right approach, which is to make, like, again, as few tweaks to this genome as possible — just so you get these basic features — and then let nature do the rest. Get to, you know, 5, 10 generations of these and that will refine it.

      Hanne: I love when you say you realized the idea isn’t why — how crazy this is to do it as actually, like, “Well, it’s actually not that crazy.” The reason is, like, why wouldn’t it work, right?

      Sonal: I love it too.

      Ross: Right. Right, right, right.

      Hanne: Do we know exactly what the woolly mammoth was? Do we know exactly what we’re aiming for, or are we guessing?

      Ross: We have used several DNA fragments to sequence, like, the entire woolly mammoth genome. However, we are not trying to make — so, I’m speaking out of two corners of my mouth here because I’m saying we’re gonna manufacture mammoths, but what we are actually gonna do is manufacture a furry, fatty Asian elephant. Like, we are not aiming…

      Hanne: A mammoth look-alike.

      Ross: …for the original genome, for the exact genome of the original mammoth. We’re just looking to remodify Asian elephants.

      Hanne: An Asian elephant with the characteristics of a woolly mammoth in certain key areas.

      Sonal: Just to give some textural feel, you described that Church and his group are adding cold-resistant hemoglobin, a full-body layer of insulating fat, they’re shrinking the ears.

      Hanne: Why are they shrinking the ears?

      Sonal: Why are they shrinking the ears?

      Ross: Good question. Well, imagine, you know, in the Arctic, you get, you know, 70 below during the winters.

      Hanne: Frostbite.

      Ross: The African elephant has these huge ears, and those are not needed in the Arctic. And then you said cold-resistant hemoglobin. I wanted to call it antifreeze blood.

      Sonal: Like a new version of True Blood, like, “Drink this antifreeze blood.”

      Ross: That’s right. And they wouldn’t let me get away with it.

      Sonal: Hanne, you asked an amazing question about, you know, is it actually doing it from truth or not, but is there a truth? Because you also pointed out, we have this dead DNA problem. Like, you think of DNA as this thing that lives on for ages and eons, but in fact, those DNAs decompose and [are] not really available even to draw from.

      Ross: That’s right. One reason that we’re looking to just modify Asian elephant genomes instead of, like, doing the Jurassic Park style, like, “Oh, we found it in the amber,” is that, look, even after a few thousand years, DNA gets really decayed, and by cosmic rays, and by microbes, and by any number of — nature is really — you know, the universe is a really harsh place. Oh, yeah.

      Hanne: So it sounds like you’re sort of saying like it almost doesn’t matter. As long as an elephant can live there, it’s okay. But once we start giving them these different — and we’re introducing a new animal into this very complicated ecosystem environment, like, does it maybe matter that they’re not exactly the woolly mammoth?

      Ross: My view is that it’s worth what will probably be some considerable suffering on the part of the first few, if not more, generations of these mammoths. And like, I’m alive to that, and I actually try to talk about, in particular, the social suffering. I mean, elephants are really social animals. They hang out in matriarchal herds. Their grandmothers are around, like, teaching them, you know, all of these behaviors. They grieve their dead. They have, like, a really rich communication with, like, you know, these little rumbling sounds, many of which are inaudible to the human ear. They’re some of the most social animals on the planet.

      Hanne: How do we even know, you know, these unformed, untaught — these poor difficult new things, dropped into this new landscape?

      Sonal: And by the way, all at the same age.

      Hanne: How do we even know they would know to do what we want them to do? I mean.

      Ross: I suspect that — have you ever seen in the zoo, they have the guy who gets in the mama tiger suit, you know?

      Hanne: Yeah, yes.

      Ross: I think there might be something like that happening early on. I mean, I can’t imagine.

      Sonal: We think of these as purely biological things, and we forget that there’s a transmission of culture that has to happen as part of it. And in fact, even the language you use in the piece — I actually was a little taken aback. You have this language, and it’s beautiful. As an editor, I’m like, “Oh, gorgeous diction.” You talk about how we sculpt them to survive the winter but let natural selection do the polishing. It felt more like playing God, just bluntly. Like, it’s like creating a Galatea clay. I don’t know, Pygmalion and Galatea, like, you know, whatever.

      Hanne: Well, I think, yeah, it reminded me — it feels to me like making a golem, kind of, right, because we’re shaping the outside, and we’re not doing any of the — and when you’re describing all the complexity of, like, you know, the biology of the gut to eat the tundra and, like, all of that complicated, you know. And then we’re just, like, shaping this stuff out, the exteriors and then plopping them down.

      Ross: Well, the other thing — I mean, I think this really gets to one of the philosophical tensions that I wanted to confront, to your point about playing God. Another thing that’s like playing God is removing 95% of the megafauna from the surface of the Earth.

      Hanne: That’s right.

      Sonal: Yes.

      Ross: We have natural human biases around things like gene editing that, like, get us all prickled and, like, “Oh, we’re playing God.” But in fact…

      Hanne: But in fact, we’ve been editing everything.

      Ross: …we have this tremendous effect on the Earth.

      Sonal: So let’s break down some more of the science on playing God. So we talked about CRISPR, the gene-editing tool, and let’s talk about the genes. So we described some of the characteristics and features that we wanna add, but by my count, there are 95 genes to do the job. 15 that were completed, 30 that are being tweaked, and he says George Church was guessing that we need maybe 50 more.

      Ross: He actually was saying, you know, a total of 50. Beth Shapiro, who I regard as sort of the world expert on this stuff — she was, like, you know, “Not so fast. You have to see what those changes do to the rest of the body, and how they interact with each other.” So like, sure, maybe 50, but it’s too soon to say.

      Sonal: Right. Well, the other thing that I found very fascinating, especially in the tales of that recent news about the artificial womb in an animal being able to be incubated, is that you essentially grow these mammoths in an artificial womb. So what’s that process?

      Ross: Yeah. And I’m glad you brought that up, because actually, that is the most science-fictional aspect of this whole thing.

      Hanne: That’s the biggest stumbling block.

      Ross: That’s the biggest leap, yeah.

      Sonal: Interesting.

      Ross: Gene editing, you know, it’s a known technology, it’s a matter of trial and error. It’s like, “Let’s, you know, keep spitting out embryos with, like, different changes, and eventually, we’ll get there.” Growing an embryo, especially this is the animal with the longest gestation period.

      Sonal: Which is what, 22 months or something?

      Ross: Two years. Yeah, yeah, almost two years. That’s right. And it’s, you know, 200 pounds at the end of it, and you’re gonna do all that, like, really complex fine-tuning, maternal fine-tuning, like, hormonal work in this huge closet-sized tank. Like, that’s more than 10 years away. George Church thinks that you can make a mammoth, like, genetically, within five years. And he said to me, “Just like there’s uncertainties on the pessimistic side,” like, “Oh, actually it’ll take 20,” he’s like, “It could take shorter, you know.” But growing an actual elephant, a furry elephant, in a tank — we’re not there yet, technologically. That is a thing that, it’s like, no one is working on even as hard as these guys are with the gene itself.

      Sonal: I hear you when you say it’s the most science fiction of this whole piece, but when I heard the recent news about the artificial womb, it actually gives me great hope, because you think about all, you know, the collateral good things that come out of this kinda science and work. Like, will we be able to have true artificial wombs for human beings as a result of this work, or other things that we can essentially let women have kids? Like, that’s just a beautiful idea to me, that we can actually manipulate that on some level.

      Ross: It’s completely lovely. But just to put that in context and to illuminate the challenge, if you were to make it analogous to human beings, women have, like, a 40-week gestational period. These are, like, preemie lambs. Like, they were born at, like, the equivalent of 22 human weeks. And they stuck into these artificial wombs, and they were able to go to term.

      Why grass was essential to early humans

      Sonal: Let’s go back to breaking down the characters one by one. We need to talk about grass. You mentioned that Ice Age is actually really a grass age and, by the way, that the formal name of Ice Age is the Pleistocene Age. I actually didn’t connect [that] all three of those things are actually the same thing.

      Hanne: Is it exactly what we think of as the Ice Age?

      Ross: It is the Ice Age, it is. So it’s 3 million years, and the really interesting thing about it is, it’s kind of, like, the nursery period for human beings. Like, this is where we sort of, you know, discovered fire, learned to harness fire, developed language, developed advanced tool use, and then, all of a sudden, we kinda pop up, history starts, what, like, you know…

      Sonal: Accelerates out of there.

      Ross: …5,000, 6,000 years ago where you have, kind of, genuine writing. But all those behaviors really incubated in the Ice Age, so I’ve always been kind of fascinated with that period.

      Sonal: And timescale-wise, that ended 12,000 years ago.

      Ross: Yes.

      Hanne: Can I just have a moment of fan mail here?

      Ross: Oh, God, please.

      Hanne: I love when you looked at one blade as, like, this little soldier fighting this grand army, you know, of the wages of, like, the planet.

      Ross: I went down deep in this ice cave with Nikita, the son in the story, like, walking around in a geode. Almost every surface is covered, you know, with sparkling ice. And we get to, like, the bottom of this little chamber, and you know, he sort of, like, scratches at the ice wall, and he pulls out this, you know, pale dead blade of grass from the Ice Age from 30,000 years ago. And at the time, I will confess to you guys a little sort of writerly craft. I watched…

      Sonal: I thought you were gonna confess fear, because I was thinking about that whole thing, and I was, like, “Holy fuck, claustrophobia, cave, freak out, cold.”

      Ross: Totally, totally. Fair, fair. So going into the piece, I really thought that the kind of reigning mythology that people will have in their mind reading this article is Jurassic Park. And so, how can I, kind of, subvert that, right? When they’re kind of explaining how they do the resurrection of these dinosaurs, there’s a moment where they’re in a cave, and they hold up to the light this amber, and there’s an ancient mosquito trapped in it. And I thought, like, “Is there a way I can get an image like that?” And so, then, at the bottom, when he pulls out this piece of grass, I was like…

      Sonal: Here it is.

      Ross: “That’s my zip line into the deep past.” I’ll have to admit, I had always been much more romantic about forests than grass going into this piece. Sergey was talking about grass and its importance in the rise of humans, in particular. That really captured my imagination, and was an idea that I felt like was not out there in the world.

      Sonal: And what is that? What is the connection between grass and humans?

      Ross: Well, grass is, like, kind of the newest big plant-based biome on the planet. Like, forests have been around for, you know, 300, 400 million years, and grass is, like, less — well, big grasslands are less than, you know, 60, 70 million years old. And they’re really neat, like, grow really fast. They just, like, erupt out of the earth, and they make food very easily for animals. And they’re not — a lot of them are not afraid of being eaten. They love to be eaten. So you have trees, you know, well, like, or other plants, will invest all this energy into thorns and into poisons because they’re, like, “Get away from me.”

      Hanne: “Back off.”

      Sonal: To repel people from eating them.

      Ross: “I don’t want you to eat me. Let me do my thing. I wanna grow.” And grass is, like, “Eat me. Eat me. Eat me. Eat me.”

      Sonal: They’re sweet. They’re like, “Yeah.”

      Ross: “And just poop me back out, so then I can grow even more, and you can eat me again, and you just go, go, go.”

      Sonal: Into this feedback loop. You have this line actually — had so much packed into it. By allowing themselves to be eaten, they partner with their own grazers to enhance their ecosystem’s nutrient flows.

      Ross: Yes. The animals poop them out, and they poop — you know, the great thing about poop, while we’re, you know, talking about things that we didn’t know were so great, like grass, is that it’s really sort of warm and kinda seeps into the earth very quickly, and it’s been processed by microbes. It’s like, kinda, you know, juicy.

      Hanne: It’s ready to go. Natural fertilizer.

      Ross: Yeah. It’s just fertilizer, right? We know, right, so what do we use for fertilizer? And so, it makes these grasslands just, like, cycle, cycle, cycle really quickly.

      Sonal: I agree, this idea of the grass is so counterintuitive, and I first came across it in “Sapiens,” and one of the things he says is that humans tamed — created humanity, because it allowed us to use wheat to, like, drive our lives, and there’s all these different forms of grass that exist now. You’re describing rice, wheat, corn, sugarcane.

      Hanne: I thought it was really interesting how, like, this is a portrait of all these, you know, cutting edge, sort of, science and tech discoveries and capabilities. And we’re using it to, like, reach deep into our, like, no longer accessible past. Like, you described this moment of solastalgia, right, like, this yearning for what once was. That’s kinda part of the human condition.

      Sonal: And by the way, solastalgia, as in an existential grief for a vanished landscape — because that was the first time I ever heard that word.

      Hanne: Yeah.

      Ross: Yeah.

      Sonal: I didn’t know what the hell that was.

      Hanne: No. Me, too. Yeah.

      Ross: Yeah. It’s a very minor philosophy. Yeah.

      Hanne: Good. I was hoping you would define it. Yeah.

      Ross: Yeah. So I’m really drawn to stories that show humans interacting on long timescales, which is a thing that I think we’re doing more and more now.

      Sonal: By long timescales, you mean like cliodynamics or just anything that’s, like, the arc of history? What is that?

      Ross: Yeah. I mean, like, when we think about what it’s going to mean to be human beings now and in the future, that we’re taking into that context 10, 20, 30, 40 millions of years into the past, and perhaps 10, 20, 30,000 years into the future. And this is, I should, again, give a shout-out to Stewart Brand, who obviously has had many fertile thoughts along this path, but…

      Sonal: Stewart Brand who is the father of the “Whole Earth Catalog” and now runs The Long Now Foundation.

      Ross: Yeah. But this idea of looking at our existence in a way that really zooms out from our current moment, which is certainly a relief in this particular historical moment we find ourselves in.

      Sonal: There’s this interesting juxtaposition between past and present that’s so fascinating, both mechanically and then historically. But even down to some other random details, like, you mentioned — the first most popular Arctic station besides this one is the one in Alaska, and that’s one place that Amazon Prime delivers to.

      Hanne: I know.

      Ross: I know.

      Hanne: I totally was struck by that too.

      Sonal: I was just, like, “What the?” That was, like, “Wow.”

      Hanne: It was unbelievable.

      Ross: Didn’t that sound awesome?

      Hanne: Yeah.

      Sonal: That is so awesome. And it’s so funny because the other Arctic station is, like, “Okay, we don’t have Amazon Prime, but we have alcohol. Lots of vodka.”

      Hanne: Like a little competitive, like.

      Ross: They really go all-in on it too. The town that’s close to Pleistocene Park is, like, a really depressed mining town, and so I was wondering, like, “You must have poachers.” And he said, “Well, no.” You know, they hunt in all the forest around it, but they don’t hunt in the park. And I was, like, “Well, why not?” And he said, like, you know, personal relationships. And then he says to me, like, you know, “When the leader of the local mafia died, you know, I gave the opening remarks to his funeral.”

      Sonal: I mean, it is an interesting thing about science meeting society. Like, when you have science not in a lab and playing out in the physical environment, you are gonna bump into things like cultural realities, poachers. One of my favorite things I’ve ever done in my life was go to this Jurassic Park of India. It was just a few years ago that I went. It’s called Balasinor. And it’s the world’s most ancient enclave of dinosaur eggs.

      Hanne: Whoa.

      Ross: Whoa.

      Sonal: Yeah. And I’d found it by accident, because I was doing, like, some local research. And I rented a special truck. It took us forever to get there, even though it’s so close, because it’s on these down, windy roads. And the thing that was so amazing is you see these dinosaur eggs fossilized in the rock, but all the dinosaur pieces — the whole way that Balasinor was found is because some local women in huts nearby were using it for plates and bowls.

      Hanne: Oh, my gosh.

      Ross: Oh, my gosh.

      Sonal: And they had no idea of the value. And they actually then put it on the market, some scientist came across it, and then all these scientists descended. But you have the government, you have the locals, you have the scientists, and you have all these characters. And one thing that did strike me in your piece is — that was kinda left unanswered is — who’s paying for all this?

      Marketing long-term study projects

      Ross: They’ve got NSF funding, and funding from the Russian government at the moment, and they do that partly because if you wanna study the permafrost or the Arctic, in general, you need to have these various outposts. And so it’s worth their money to do that. The more interesting question even than the funding, to me, which you’re kind of getting to when you’re talking about this lovely story about the dinosaur eggs in India, was that, for this to expand. Like, Yellowstone right now, which is a thing that everyone loves, right, like, you can’t get people to say bad things about Yellowstone. People universally acknowledge it as being an amazing thing in the world. But, like, its expansion impinges on real people’s lives, you know, because all of a sudden, big predators are showing up in their backyard, etc. And so, for something like Pleistocene Park to be successful, it’s going to have to interact with and make peace with the human world on, like, quite a grand scale, if they are gonna do all of Northern Siberia, and Alaska, and Yukon, etc., etc. And that as being representative of the larger tension we have of trying to figure out how we coexist with wild animals and with the wild, in general.

      Sonal: There’s a socioeconomic component too, because you think of these towns that don’t have a lot of money to survive. They don’t have a lot of economic opportunity. Why wouldn’t you wanna sell, like, ivory, you know, from these tusks and make some money for yourself to survive or support your family?

      Hanne: Or dinosaur egg china.

      Sonal: Right. And so it’s really striking when you do think about this question of who funds it, because there’s a lot of science and money that goes into this. And there’s just a lot of tradeoffs that people have to make. And anyway, another open question is, like, this project is so radical in scheme and scope that — is anyone else doing anything this ambitious in the world anywhere?

      Hanne: Well, you compared it to one other major climate project, right?

      Ross: Oh, yeah. They’re geoengineering projects or proposals. Also, the American Prairie Reserve is another large grassland rewilding project. It doesn’t have, sort of, sexy extinct creatures to sell it or, like, a major climate change mitigation strategy to sell it, but it’s really interesting, and it’s, like, part of Montana.

      Hanne: Tell us, I would love to hear the story behind the stories.

      Ross: Funny story is, going up there — this is, like, a protected area, and so you have to get official Russian permission, not just like a regular visa, just to actually go to this region. So we get there, and I had a really good friend of mine, Grant Slater, who’s an amazingly talented documentary filmmaker. We’d kinda worked together. I knew that he would have this, sort of, deep time sensibility alongside me. And so I was really excited to see what he would do with it.

      Sonal: Visually.

      Ross: And there’s also a really interesting creative tension, being out with a filmmaker, because, like, he has things he needs to get, I had things I need to get. Anyway, I’m rambling.

      Hanne: Yeah. It’s a different kind of storytelling.

      Ross: But Grant’s paperwork, his, like, official permission had not come in on time, and so we had to, like, go get — we went and got questioned at the military base by, you know, these Russian soldiers who are, like, in full fatigues, pretty big dudes. And what was funny about it was, Grant had lost one of his suitcases in Moscow. He had to buy clothes, like, at the airport. And the shirt he was wearing during our interrogation was this shirt that said, in Russian, “Russia is a great power.”

      Hanne: No.

      Sonal: It’s like a scene out of a comedy movie.

      Hanne: And didn’t they — they went, like…

      Ross: I was devastated when he got caught. Devastated.

      Hanne: They thought he was a spy, right? Like, they were, like, “You’re obviously…” and he’s wearing this t-shirt that says Russia is great.

      Sonal: No, no. Even worse, they asked him if he was a spy.

      Hanne: No, that’s right.

      Sonal: Like, a spy is gonna say, “Yes, I’m a spy.” That’s just crazy. So just to close, I think the most striking thing about this piece, that this idea sounds so crazy at first. The thing that really struck me is that the region that you were in was once famous for beaming propaganda throughout the country of Russia, and at the same time, there’s an element of marketing that has to happen in this idea, for someone to convince other people, to drive people towards their vision, to get them to believe it.

      Ross: I’m also captured by this question of how, you know, when you have these really esoteric science projects that are tied into questions of human meaning in all kinds of different ways, how you can present that.

      Sonal: And sometimes cults of personality as well.

      Ross: And cults of personalities, and how do you, kind of, make that — I mean, something that Elon Musk is really adept at, right, taking ideas out.

      Sonal: I remember you did that Q&A with them at Aeon, a long time ago.

      Ross: Yeah. Like, he’s really good at packaging crazy-sounding ideas and, like, getting lots of governments, investors, to throw lots of money into them, while managing to keep control of them. Part of that is the narrative, right? He does hook it into, like, larger questions and existential concerns in a way that I don’t think is just manipulative, I think he sincerely believes those things.

      Hanne: And I also think a lot of it is, like, just saying, like, “This is happening now.” Like, sort of making us realize, like, “Actually, this is happening now.” You know, that’s a lot of turning it around to feel possible, basically.

      Ross: Yeah. People are working on it. It’s a thing. You can go there. Yeah.

      Hanne: Yeah. It’s a thing. Yeah.

      Sonal: Well, also that it takes time, because one of the most telling anecdotes in your piece — because you know, there’s a whole debate we don’t have to go into in this podcast nor do we have time, about climate change deniers, climate change science, what’s legit, what’s not — a whole other conversation. But what I found fascinating was that science initially rejected Sergey’s paper about the dangers, you know, in the warming…

      Ross: Of the permafrost. Yeah.

      Sonal: Right. And in 2006, the journal then asked him…

      Ross: Yeah.

      Sonal: …he didn’t have to approach them again, to resubmit it. And it was published later that year.

      Ross: Yeah.

      Sonal: And that just goes to show you, there’s also a right time for some of this.

      Ross: Right.

      Sonal: Like, there’s a readiness that has to happen. Thank you for joining the “a16z Podcast.”

      Ross: Well, thank you for having me on.

      Hanne: Thank you.

      • Ross Andersen

      • Hanne Winarsky

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Golden Era of Productivity, Retail, and Supply Chains

      Marc Levinson, Hanne Winarsky, and Sonal Chokshi

      This episode of the a16z Podcast takes us on a quick tour through the themes of economics/historian/journalist Marc Levinson‘s books — from An Extraordinary Time, on the end of the postwar boom and the return of the ordinary economy; to The Great A&P, on retail and the struggle for small business in America; all the way through to The Box, on how the shipping container made the world smaller and the world economy bigger.

      In this hallway-style conversation, Levinson and we (with Sonal Chokshi and Hanne Winarsky) touch on everything from productivity growth & GDP to the “death of retail” — to finally connecting all the dots through logistics, transportation, infrastructure, and more. How are supply chains changing? How does all this, taken together, affect the way we work? And what can — or can’t — policymakers do about it? Perhaps, Levinson argues, a lot of the improvement to our living standards really comes out of “microeconomic improvements at the private sector level rather than as a matter of great policy”. But that’s a bitter pill to swallow for those seeking solace in easy answers from governments, whether at a national or city level. Maybe it’s just a matter of managing our expectations — or resetting our clock for when the new normal begins… and ends.

      Show Notes

      • Why the post-WWII era was a golden age for economic growth [0:48]
      • The current state of the retail economy [8:43] and how the history of A&P relates to today [12:43]
      • Discussion of shipping and supply chain issues [15:50], and issues that will face future generations [27:38]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast,” I’m Sonal. Today’s episode with me and Hanne is another one of our podcasts from our recent road trip with Voices from the Ground in Washington, D.C., though this one actually takes us all around the world. Our guest is the economist, historian, and journalist who was last at “The Economist,” Marc Levinson, the author of the beloved book, “The Box,” which is about how the shipping container made the world smaller and [the] economy bigger. But this hallway-style conversation is actually a quick tour through all his books, starting from his most recent one, “An Extraordinary Time,” where we touch briefly on the topic of the golden age of productivity and beyond, to the topic of the death of retail in his book, “The Great A&P,” to finally wrapping up on logistics, transportation infrastructure, supply chains, and touching very briefly on the future of work and where government comes in policy-wise in all of this or doesn’t.

      The golden age of economic growth

      We’re so excited to have you. Welcome, Marc.

      Marc: Thank you. Glad to be with you.

      Sonal: So, those seem like really different topics. What’s the big idea that drives the thrust of your work — that kind of connects all the dots?

      Marc: I’m really interested in the connections between economics and the world we live in. A lot of my work starts out at a microeconomic level, looking at particular companies, looking at particular industries, and tying the developments there to broader trends that really affect how we live, affect our standards of living. More recently, I’ve been focusing on some of the trends in productivity growth, because I believe that a lot of the improvement in our living standards really comes out of these, kind of, micro-improvements at the private sector level, rather than as a matter of great policy. And what that means, and this is a frustration for public officials, is that there are no easy government solutions. We’ve now been through generations in which politicians and the economists who advise them said that they had the cure for productivity growth.

      Hanne: Right.

      Marc: I argue in “An Extraordinary Time” that, actually, this was what was behind the political swing to the right in the late 1970s, early 1980s, when we got Margaret Thatcher and Ronald Reagan. Because the more social democratic types of governments before that hadn’t been able to restart productivity growth. And so, voters turned to people with other ideas, but the people with the more free-market ideas proved no more successful than the people with the more statist idea.

      Hanne: What are we actually comparing to, as we’re thinking about these, like — “Well, this is not good enough?” What are we holding up as, you know, something that we’d prefer it to be?

      Marc: The end of the post-war boom and the return of the ordinary economy. The story I’m telling is that the quarter-century after the war was an unusual period of very rapid economic growth. The period from 1948 to 1973 was probably the period of the fastest economic growth in the history of the world. GDP around the world grew at more than 5% a year.

      Now, at 5% a year, something doubles in 14 years, quadruples in 28 years. So, even with some population growth, people’s incomes were growing very rapidly. People’s living standards were rising in a way that was visible to them. They were able to buy houses for the first time and cars for the first time, and send their kids to high school, and maybe even college, and we had all kinds of very rapid advances in living standards.

      Hanne: What was that due to? What was the big driving force?

      Marc: We had an unusual confluence of factors in the post-war period that people have really forgotten about now. One is that there was a great deal of underused capacity — underused resources in the economy. I’d like to remind people that at the end of World War II, we still had 3 million mules on farms in the United States.

      Sonal: Wow. It’s such a technical, post-industrial revolution time. You don’t even realize it. It’s that…

      Marc: You have millions of people, and not just in the United States — European peasants and Japanese farmers owned half an acre of land — who could be moved from very low-productivity jobs into very high-productivity jobs in the cities. And we had a lot of that in the ’50s and ’60s. So that was one big boost to productivity. We had very rapid increases in education levels, and we know that education is associated with productivity. In the United States, at the end of World War II, going to college was not common. It was — just a few percent of the population of 18-year-olds actually went on to college, and the average education level was around eighth or ninth grade. So, in very few years, governments spent a lot of money building a more educated workforce and it paid off.

      Sonal: The government was the one that seeded that, or was that just a shift in the fact that adolescents existed and childhood changed?

      Marc: No. This was heavy expenditures, building universities all over the place. Take a look at how many universities in the United States started after World War II, okay? That’s when a lot of government money started — it was no longer an elite thing to go to university.

      Hanne: And how about women entering the workplace?

      Marc: Well, women entered the workplace. The other thing I think that was really consequential in this quarter-century I’m describing was that we had the growth of motorways — the interstate highway system in the United States.

      Hanne: So, public infrastructure, like, the transportation infrastructure?

      Marc: Public infrastructure, and think about what that does. If you are a manufacturer or a retailer, that lets you sell over a wider area, lets you operate your facilities more efficiently. You don’t need a warehouse in every town. You can have one that’ll serve a large area. If you’re an employer or a worker, it’s changed the size of your labor market. I mean, in Silicon Valley, San Jose and San Francisco are now part of the same labor market, right? That wasn’t the case after World War II. These were very different cities and they were a considerable drive apart.

      And so, that creates a better fit between people and jobs, and then leads to higher productivity. They can’t be repeated. Once you’ve moved those sharecroppers to the cities to take jobs in industry using heavy machinery, they’ve moved — and you don’t have those underused resources again. There were countries around the world that literally went 25 years from 1948 to 1973 without a single year of recession. We had countries that had less than 1% unemployment back then.

      Hanne: So, how did this burst of productivity, this golden age, actually come to an end?

      Marc: Well, in 1973, we really saw a trend change. That was the year of the great oil crisis that some people may remember. What we have moved into, since 1973, is really an environment in which economic growth has been slower. The improvement in living standards has been slower, the unemployment rate, in most countries, has been permanently higher. We have not been able to recapture the very unique good times that we had in this golden age, and I think that we’re not going to be able to. What we’re experiencing more recently is actually normal. This is the way most economies work most of the time.

      Hanne: And how it worked before this golden era?

      Marc: The golden age was actually the exceptional time. It’s not normal that economies grow at a rapid pace. It’s not normal that incomes double, or triple, or quadruple in a matter of just a few years, and I don’t think we should expect that to recur.

      Sonal: I view this as analogous to child development and how a human body, an adult develops. Because there’s a rapid development that happens in the birth of a child, and then another big rapid shoot that happens in adolescence, and then you continue to grow, but it’s a little slower. And, in fact, thinking about the natural conclusion of your argument — is that that growth has now shifted to other countries like India, China, where they are now experiencing the kind of boom that you were describing that happened pre-1973. 

      Marc: That’s a great analogy. Take Japan which in the 1960s or early ’70s was growing at 7% or 8% a year, and then it downshifted, and then it downshifted some more. More recently, China went through a period where it was growing at 10% a year.

      Sonal: By the way, in China’s case, we do have to take the numbers with a big grain of salt.

      Marc: Even so, people were extrapolating and saying, you know, when China grows at 10% a year for the next century, its economy is gonna be twice as large as the rest of the world put together.

      Sonal: Right.

      Marc: But China’s not going to grow at 10% a year for the next half-century. It’s becoming much more like a normal, mature economy in which the growth rate is a couple of percent a year, and that’s all they’re going to be able to expect.

      The current retail economy

      Hanne: But it doesn’t mean necessarily that, like, in adolescence or growing as a human being, you only get one burst. These things can come in waves, there can be other, kind of, confluences to these factors. When you were talking about the Japanese farmers with their mules, and the, sort of, move towards the way automation and the move to cities increased productivity, are there any inklings that you’re starting to see of possibilities, like, with the automation we’re starting to see happen today, and maybe even with autonomous cars, new infrastructure might, you know, for city infrastructure — are there things that give you any sense of maybe a new era might be coming at some point?

      Sonal: Or just even a way to juice the body on steroids? Like, just inject some more steroids into this economy?

      Marc: You’re asking great questions here, and the answer is maybe. I think that these are things we really can’t predict. If you look at past episodes of fast productivity growth, in general, they weren’t predicted very well. For example, we had a spurt of productivity growth which translated into faster income growth in the late ’90s and the first years of the 2000s. This is the famous internet boom, you may remember.

      Sonal: Oh, we remember it.

      Marc: In 1992, no one predicted this. What happened was that there had been investments in infrastructure, there had been developments in technology decades earlier — and, finally, during this period of time, they all came together.

      Hanne: But I think a lot of people would argue that we’re in a moment like that again now.

      Marc: Well, I think that’s a question which we can’t answer. So, if you take a look at a technology, will it actually revolutionize the way certain industries work? I don’t exclude the possibility, but you have to keep in mind that there are also a lot of complications. You’re seeing this right now as we go through this rather brutal shakeout in retailing. Yes, everybody knows that you can order goods on the internet. That’s not news these days, okay? But the reality is that for a lot of retailers, there’s a problem here, because they’re maintaining an internet business, they’re also maintaining a retail store business, because some customers want that. So, in some cases, their costs have gone up. They have not become online retailers — they have become bricks-and-mortar/online retailers.

      Sonal: Right. And they’re <inaudible> for the online sometimes.

      Marc: They’ve got multiple channels that they’re having to service, and that’s actually made their operations less efficient in a certain way.

      Sonal: I would actually say there’s a flip side to this, though, again — which I think is really fascinating. Because when you think about the internet economy, birth of Amazon — which is, let’s face it, the behemoth in everything, the everything store, the everything everything. And they recently, as we know, started doing physical brick-and-mortar bookstores. The difference is that they started online and they went into physical, using data to help stock and think differently about how to create their store in an internet-native way in the physical world. So, I also wonder if while the death of retail might be on the horizon, if after that there might be an entire new post-boom — a new boom around retail that’s completely reshaped by new technologies. We don’t know.

      Marc: That’s entirely possible. But just to give you something to think about, Amazon’s problem in terms of getting its books into its physical stores is entirely different from its problem getting its books ordered online to you, the customer, okay?

      Sonal: Yes.

      Marc: So, now it needs a different kind of logistical system. It needs to figure out how to distribute to retail stores like the ones it, apparently, is building. That’s going to have a lot of costs attached. It may have some inefficiencies attached, at least while they’re developing it. So, my point is to say that the path of — you know, sometimes people who are involved in the tech industry, kind of, get very romantic about how quickly these great technologies are getting absorbed. But, in reality, life is messy, and some of these technologies take a while to be used efficiently, and some of them will never be used efficiently.

      Sonal: That’s right. There are more failures than there are successes. There’s no question about that.

      Hanne: So, does it remind you at all of the, sort of, death of the supermarket that you talked about in your book, “The Great A&P?”

      Marc: In “The Great A&P,” I was writing the history of what was, for about 50 years, the largest retailer in the world. People forget this now, but The Great Atlantic & Pacific…

      Hanne: Is that what A&P stood for?

      Marc: Yeah.

      Hanne: I had one in my town. I didn’t even know that.

      Marc: Yes, it was The Great Atlantic & Pacific. It was so named in 1869 for the transcontinental railroad.

      Hanne: Wow.

      Marc: And at one point, it had more than 16,000 stores in the United States, so it was a behemoth. It was the Walmart of its day. But one of the things that kept it so vibrant is that it remade itself continually, because shopping trends change, consumer expectations change.

      Hanne: From what to what?

      Marc: It started out as a seller of coffee and tea and spices. It made itself into a small grocery chain. And then, in 1912, it developed the idea of having an economy grocery chain, which is to say, it would have a very bare-bones store and sell products much cheaper than the competition. And that’s what drove its growth in a small period of years. It integrated vertically, so it made its own chocolate, its own macaroni, canned its own salmon — and then had a huge network of manufacturing plants, and, again, we’re in the 1920s here. And then it started building supermarkets. It was not the innovator in any of these things. A&P did not develop the idea of supermarkets, but once it saw how supermarkets would work and how they would fit with its business, it started building supermarkets all over the place, and by the end of the 1930s, was the biggest supermarket operator in the country.

      Hanne: So, what ended up being its downfall?

      Marc: The company stopped innovating. The company stopped remaking itself. It was big, it was fat, it was happy. The two brothers who had controlled it for decades both died in the 1950s, and it was then run by people who had been with the company for decades, and whose idea was to preserve it rather than to keep it changing.

      Sonal: You know, the beat that keeps coming up in this is that, basically, the tension between this idea that you can innovate but then you get too good at what you do, too comfortable, too complacent. Okay, so what’s the big, then, lesson or takeaway, you know, from your work on “The Great A&P” to this narrative around the death of retail today?

      Marc: Retailing is full of dead bodies. People like to talk about how unfair competition is sometimes, because the big companies have more power than the little ones. But when you are a big retailer, you can’t change so easily. Okay, if you own one store and you think you need to do something else, you go in there with a hammer and some plywood and you can do it. If you own a thousand stores, you’re stuck. Okay, you’ve got your locations, you’ve got your product line, you’ve got your brand name, and you can’t change easily. It’s a really difficult situation, and so a lot of stores end up dead.

      Sonal: It’s the innovator’s dilemma, classic case.

      Marc: It really is.

      Shipping and supply chains

      Sonal: So, another theme that’s come up and that connects all the dots with this entire conversation — that this post-boom world was one of the drivers — was this, like, rapid development of infrastructure. Amazon exists because of logistics and infrastructure. Like, innovations and being able to ship things and deliver things fast. You know, we talk about the supermarket and the growth of suburbs around railroads and transportation. Transportation on logistics and infrastructure is, like, the thread that connects and drives all economies. So, let’s talk about “The Box,” which is all about logistics and infrastructure in the form of container shipping.

      Hanne: You chose one very specific thing in “The Box” to talk about. It had this massive effect on the global economy. Give us a little bit of sense of what that story was like. I remember Tim describing, when he pitched your book, this incredible scene of just the giant mountains of peanuts in the ships. Like, how quickly did shippers see this possibility and start using it? Was it fast or slow?

      Marc: Well, let me give you just a quick history here. The idea that you could save money by shipping goods in containers came along in the 1700s, okay? This was an old idea, and nobody had ever figured out how to make money out of it. Because what would happen was that, well, you would make a container out of wood and nails, and you’d put your goods in it, and then at the other end of the trip, somebody would break the container apart and use it for firewood. That was a pretty inefficient system, and nobody ever found this to be viable. It actually cost more to ship goods and containers.

      What made this whole thing work was the arrival of a guy named Malcolm McLean, who was a trucker — so he didn’t come from the shipping industry — and he understood that what was needed was not particularly a container but a new system for moving freight, okay? And the container was just a piece of what…

      Sonal: It was just a container.

      Marc: It was just a container, and a lot of people back in the ’50s and ’60s who were in the shipping industry were very enamored of their ships, and they thought they were in the shipping business. And McLean’s basic position was, nobody cares about your ship — they just wanna get their goods from here to there, and let’s design an efficient system for doing that. So, shipping containers first came into use in the United States in 1956. They started being used internationally across the Atlantic in 1966, and the industry was pretty substantial by the 1970s. By that time, most of the older vessels had gone out of service. But it was really in the 1980s when modern logistics took off.

      There were a couple things that happened. You had, in this country, freight deregulation — which meant that you could actually sign a single contract to import products that would cover delivering the goods to a port and moving them inland by rail to a final destination, or having a truck pick them up and move them to a final destination. So, you could actually integrate all these modes of transportation and have some assurance that the goods would get there. And then you had improvements in communications. This was referred to as electronic data interchange.

      So, all of a sudden, it became possible to run an international supply chain in the 1980s. Now, you could send instructions across the ocean quickly about how you wanted something shipped or how you wanted something made. And so, this innovation — the container — that had really come about in the ’50s, started to make a substantial difference in the world economy in the 1980s when we had the birth of modern supply chains.

      Sonal: The most fascinating thing to me — the idea that astounded me most about the box was the idea that the containerization of moving goods allowed things to travel multimodally. That because of that — this modularization — you could now bring things across ship, to train, to plane across the world. And that is, like, a really eye-opening idea. And it’s really interesting that you referenced the EDI, because the analogy that I was thinking of was actually packets and moving data across lines, like the ethernet. And that packetization of data also led to this thing where you can move things across phone lines, ethernet lines, other computer lines, broadband, etc., and essentially reassemble them at the other end.

      Marc: That’s a good analogy.

      Sonal: That’s a really, really mind-blowing idea. So, I guess the question I have is, what’s happening next and now that you think is interesting in the next evolution of supply chains that is along these lines?

      Marc: Well, there are a couple of things that are going on in supply chains, and they’re not necessarily good. International trade and manufactured goods actually grow more slowly than the world economy for the past six or seven years. That’s a big reversal from the previous trend. Why is that? One of the reasons is that supply chains have become less reliable. The ship lines went out and purchased very, very large vessels, and I’m sure your listeners have seen these vessels can carry…

      Sonal: I’ve actually seen them first-hand, because I went to the Panama Canal, and it’s incredible.

      Marc: Okay. The Panama Canal doesn’t handle the biggest one.

      Sonal: That’s right. Because they’re actually rate limited by the Panamax ships.

      Marc: The biggest container ships now at sea can carry more than 10,000 truck-size containers.

      Sonal: Wow.

      Hanne: That’s amazing.

      Marc: These are enormous vessels. So, what has happened? Well, imagine you’ve got a port, but instead of having a ship carrying 2,000 containers showing up every day, now you’ve got a ship that carries 10,000 containers showing up once a week. You’ve got a mess on your hands, because you’ve got this enormous load of traffic which you need to get all of these containers out of the port.

      Hanne: It’s too much.

      Sonal: It’s like a bottleneck congestion.

      Hanne: Yeah.

      Marc: Yeah, that’s right you’ve got a bottleneck. And this has come at a time when growth and trade has been pretty slow, so there’s considerable overcapacity in the industry, and even so, the reliability has fallen. So, what you’ve seen is, actually, manufacturers and retailers contracting their supply chains. They would like to make things closer to where they’re used, because they think there’s less risk.

      One of the things that I think happened in the growth of these international supply chains is that companies paid a lot of attention to cost. They said, “You know, our hourly labor cost in China is a lot cheaper than it is in Detroit.” They didn’t really pay much attention to risk. And risk is a cost factor. There were a number of U.S. companies that failed or came very close to failure because of supply chain disruptions. Key merchandise wasn’t available when they needed it for their factories or for their store shelves.

      Sonal: I mean, this is the story of hardware startups. The problem isn’t that they can’t plan out, and predict, and build, it’s that they need to lock down that supply chain inventory at the right time. But, yet, they have that issue — that they don’t know how many products their customers are gonna buy, so they don’t know how much to make. So, there’s a sort of chicken-egg problem.

      Hanne: It’s the same thing in book publishing.

      Sonal: Yeah, yeah.

      Marc: So, you’ve got the container ship lines that essentially created their own crisis.

      Sonal: It’s like a success disaster of sorts.

      Marc: They got bigger and bigger ships, because that was more efficient for their purposes. Their own costs running ships went down per container as the ships got bigger. They didn’t devote too much thought to the problems of the ports, or the railroads, or the truck lines, and all of them have had a lot of difficulty coping with this flood of containers. And so, I think one question facing this industry going forward is whether these long-distance supply chains will continue.

      Sonal: Well, I’d like to ask a question, though — is that necessarily a disaster? Because isn’t that also the inevitable sort of cycle of things aggregating, un-aggregating, lengthening, and contracting, etc.? And also, in that same context, one of the arguments I’ve heard for — there’s actually advantages to shorter supply chains. For example, in the case of, like, hardware and software innovation, there’s this rapid iteration and back and forth that happens.

      So, if you have a components manufacturer in Mexico, and you’re designing a chip or some piece of hardware, you could rapidly iterate on your designs without the long delay that happens when you have a big time difference and a bunch of other logistical issues with someone, you know, doing the same thing in China. So, there’s some argument that it’s actually not a bad thing, because it actually speedens innovation almost in some cases.

      Marc: In some cases, it may speed innovation. In general, I think that manufacturers and retailers are expecting that it’s going to reduce risk. Another trend that you see is that many manufacturers and retailers are now looking to multiple sourcing. Now, in many industries, it’s cheaper to have a single source, right, because you’ve got huge economies of scale. One factory makes a ton of stuff, and that’s great so long as it’s working.

      Sonal: It’s cheaper because of the China stuff, right.

      Marc: It’s cheaper, but maybe it’s worth paying a little bit more and have an extra warehouse. We’re seeing a lot of that now. We had, for example, a work stoppage out at the port of Los Angeles — actually, the West Coast ports in general — in the early part of the century, a lockout by the port employers. How did that affect companies?

      Sonal: With a lockout, you mean, like, it was, like…

      Marc: They locked out the union workers as part of a labor dispute.

      Sonal: Oh, right, right, right.

      Marc: And so, a lot of companies said, “Well, maybe we ought to redirect some of our traffic to ports on the U.S. East Coast.” Okay, so they still send their goods to Los Angeles, or Long Beach, or Oakland, but they also send a portion of them now to Savannah or New York, because they want to have options. They want to not have the risk that their supply chain will be shut down.

      Sonal: There’s another fascinating analogy with the digital world here, because it reminds me of the time of the early days of the internet, when as the internet became super popular and more multimedia started coming online, there were tremendous bottlenecks in data traveling through pipes. And so, they had to figure out new methods to essentially reroute and decentralize it from these central choke points. So, it’s kind of a fascinating thing.

      And it also now, by the way, explains when I was in Panama, I was a little struck by this thing where every single ship that goes through spends a day being inspected before you can even put it through, and it costs, like, $1 million per ship to put it through or some — I forgot the amount, but it’s some significant amount. And it just blew my mind, like, there’s so much extra work, but now I understand, because if one ship bottlenecks that entire thing, nothing gets through for, like, the entire day, and that’s a huge blockage.

      Hanne: Right. You gotta make sure this is gonna get through.

      Sonal: Yeah, exactly. It’s kinda fascinating.

      Marc: So, what we’re seeing in container shipping now is a lot like what we saw in the United States with the railroads in the 1870s, 1880s, 1890s when many railroads went bankrupt. We went from a country that had hundreds of railroads, each a few miles long, to a relative handful of large railroad networks. We went through something similar when we had airline deregulation, starting in 1978.

      You may remember, we used to have lots of regional carriers around the country, had a number of national airlines, and only two international airlines that were heavily protected by the government. Now there’s a lot more potential competition, and, of course, the carriers have dealt with that by merging. So, you’ve actually now got a situation in which you’re supposed to have cutthroat competition, but they’ve tried to find a way around it by merging and reducing the number of airlines.

      Hanne: It’s the inevitable cycle.

      Marc: We’re headed in the same direction with container shipping now. Many container carriers are in financial distress, a lot of them have merged into the big carriers. There are now, probably, three so-called alliances of container carriers that kind of dominate world trade. So, we may be in an environment in which there are few enough players that they’ll be able to have a better handle on prices on shipping rates, and that will mean less competition. That will be good for their shareholders. It probably won’t be good for shippers.

      Future of the world economy

      Sonal: Okay, so to wrap up, we started this with your view that connects the dots between all of these books, and you have this perspective of this economist-historian. And, you know the question is — is it good to know that this is an ordinary economy, or are we just talking about cycles of things that are just gonna inevitably decline and grow? How do you know it’s just not a typical waning, and that this actually really is something different?

      Marc: I think this actually has a lot of political implications. For decades and decades, we’ve trained people to believe that the government can provide a very steady income, can provide low unemployment, can provide rapid economic growth, and I think the government’s ability to do this is limited. What we’re seeing, I think, and not just in the United States, is somewhat of a crisis of expectations.

      Hanne: It’s a reckoning.

      Marc: If you take a look at what’s going on now in Europe, or in Korea, or in Taiwan, people expect more of their public officials than their public officials can deliver. People want their incomes to grow quickly. Their public officials promise, “Yeah, we’ll bring back the good old times. We’ll make your incomes grow quickly.” But, in reality, we’re in a normal age in which people’s living standards rise slowly.

      Hanne: I feel like while you’re saying this, all I can think is, like, “Well, only really time can teach us that that’s not…” You know, because I’m trying to think, like, “Well, what can we do to reset those expectations,” but we can’t really. It’s just time and not…

      Sonal: Understanding, possibly?

      Hanne: Well, and not having the same growth, right, getting used to not having the same growth, which is sort of disheartening, when you think about it.

      Marc: It is. On one level…

      Hanne: Because my question is, like, so how did we move beyond that?

      Marc: On one level, it’s disheartening. On another level, I think we have trained people to believe that the government can deliver things it really can’t deliver.

      Hanne: So, how do we start to undo that?

      Marc: There is a question about how the available income is distributed, which is really quite separate from the question of how fast productivity is growing, how fast the economy is growing. And I think we have to have a real discussion about how income is being distributed and how automation is going to affect our workforce.

      Sonal: This is top of mind for everybody, including us, our guests, everybody.

      Marc: And I bring this up not really in an economic sense, but almost more in a psychological sense. There’s a lot of concern about where the jobs are going to be for people in the future. I’m not too concerned about that. I’m pretty confident that we’ll have ways in which people can earn livings. But so many people get some degree of satisfaction from their work, and if what we’ve got is a world in which people are doing part-time work, occasional work, unsteady work…

      Sonal: I mean, work itself has become containerized.

      Marc: That’s right. What are people going to be moored to, in this sense? And we can do anything we want to guarantee your income, but the fact is, if all you do is wake up in the morning and watch television, because you’ve got nothing else to do, you’re not gonna be very happy. So, I think we have an issue here that really goes beyond economics to…

      Sonal: To finding meaning.

      Marc: Finding meaning in our lives as human beings.

      Hanne: To the human condition, yeah.

      Sonal: One last thing. You talk about a crisis of expectation, but the reality — and this is actually another thing that connects the dots — is that these problems are multifactorial, multimodal. There’s crisis expectations around the world, and they’re playing out in ripples and waves in different ways across, you know, France, Europe, the U.S. in so many different ways. So, I also wonder if there’s some change in the geographic or governance structures that we have to think about. Do you have any thoughts on, sort of, the geopolitical implications of this? Is there any, like, parting thought on that front?

      Marc: We’ve seen, obviously, a big shift away from faith in the nation-state, to people wanting power and control closer to home. I think we’ve seen over the centuries cycles in that — that that sort of comes and goes. Will city governments be able to deliver satisfaction in a way that national governments can’t? I’m not convinced of that, but in some issues, some areas related to people’s quality of life, they can be very important. I think another issue that we face is that there are things we know that will improve productivity over time.

      Sonal: Yeah.

      Marc: We can’t predict how that’ll work, and we still need to make those expenditures. So, for example, we know that improving education levels is, in general, good for a country’s productivity. So, we need to invest in education, but can we say if we spend an extra billion dollars in education now, that it will improve productivity three years from now? We can’t say that. We’re doing this somewhat on faith.

      Sonal: Especially if the skills and jobs of the future change.

      Marc: Well, that’s correct. We know that as a general proposition, it’s been important for economic growth that we’ve had scientific research going on. Does that mean that if we put more money into scientific research now that we will be able to benefit from the consequences at any predictable time in the future? The answer to that is no.

      Hanne: Right, and these are the reasons exactly why we tend not to make these decisions.

      Marc: That’s exactly right. That’s exactly right.

      Hanne: Or we can’t.

      Marc: These are very tough choices, because you’re asking for our tax money to be spent, and you really can’t promise the return.

      Sonal: It’s like investing in the future, difficult to do.

      Marc: Exactly. Exactly.

      Sonal: Well, Marc, thank you for joining the “a16z Podcast.”

      Marc: Thank you so much for having me. It’s been great fun.

      Hanne: Thank you.

      • Marc Levinson

      • Hanne Winarsky

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Companies, Networks, Crowds

      Erik Brynjolfsson, Andrew McAfee, Frank Chen, and Sonal Chokshi

      Is a network — whether a crowd or blockchain-based entity — going to replace the firm anytime soon? Not yet, argue Andrew McAfee and Erik Brynjolfsson in the new book Machine, Platform, Crowd. But that title is a bit misleading, because the real questions most companies and people wrestle with are more “machine vs. mind”, “platform vs. product”, and “crowd vs. core”. They’re really a set of dichotomies.

      Yet the most successful systems are rarely all one or all the other. So how then do companies make choices, tradeoffs in designing products between humans and machines, whether it’s sales people vs. chatbots, or doctors vs. AIs? How can companies combine the fundamental building blocks of businesses — such as network effects, platforms, crowds, and more — in a way that lets them get ahead on the chessboard against the Red Queen? And then finally, at a macro level, how do we plan for the future without falling for the “fatal conceit” (which has now, arguably flipped from radical centralization to radical decentralization) … and just run a ton of experiments to get there?

      We (Frank Chen and Sonal Chokshi) discuss all this and more with Brynjolfsson and McAfee, who also founded MIT’s Initiative on the Global Economy — and previously wrote the popular The Second Machine Age and Race Against the Machine. Maybe there’s a better way to stay ahead without having to run faster and faster just to stay in place like Alice in a tech Wonderland.

      Show Notes

      • The basic economics of networks and the concept of complements [1:08]
      • Discussion of whether “the firm” will survive [10:08], and our inability to simulate all possible outcomes [15:34]
      • The ability of crowds to problem-solve [20:47], with caveats related to biases in AI [23:34]
      • Advice for businesses today [30:45]

      Transcript

      Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today we’re doing one of our book podcasts around the new book just out, “Machine, Platform, Crowd.” The authors previously wrote the popular book, “The Second Machine Age.” And before that, their book was “Race Against the Machine.” Sensing a bit of a theme here. So in this episode, we cover those themes, first starting with a bit of Econ 101 around network effects, complements, and other key concepts. Then we discuss how this all plays out organizationally, especially given trends like machine learning, blockchain, and crowds — and tackle the tricky question of whether networks can replace the firm. And where are we in the classic question around the future of the firm? And finally, what can companies do more concretely?

      Frank Chen joins the conversation in between, as well, to share his perspective on what he sees, given his role as head of investing and research at a16z. But our main guests on the episode, both from MIT, are Erik Brynjolfsson and Andrew McAfee, who I’m gonna call Andy, is that okay?

      Andrew: Otherwise, I’m gonna mistake you for my mom.

      Sonal: Good, I don’t wanna be mistaken for your mom.

      Andrew: That would be weird.

      Sonal: I’m way too young to be your mom. We kind of go way back in the sense that I met you years ago and…

      Andrew: Not as far back as I go with my mom.

      Sonal: No, no. Let’s be very clear about that.

      Andrew: So, you and I will do Andy. Okay?

      Networks and complements

      Sonal: All right. Good, we’re doing Andy. So, this is your third book together. The real thrust of your work is that this is unprecedented in the speed at which we’re changing and what the effects are. And I think a great theme for this conversation is to sort of break down how those changes are going to play out, and where they’re happening.

      Andrew: Yeah.

      Erik: Yeah. Well, but let me just push back on that first part a little bit, because in Silicon Valley, everybody agrees with that. And we agree with it. That’s very clear. But we were reading people who didn’t. One of the things that got us writing our first book, “Race Against the Machine,” was there were people who were talking about “the great stagnation,” and how there were no good inventions anymore. Nothing good was invented. In particular, Tyler Cowen, he was spot on that median income had been stagnating. And that was kind of troubling for us. Because, you know, I had been taught the slogan that productivity isn’t everything. But in the long run, if we just have tech progress, everything else takes care of itself. And when Tyler showed us that evidence, we were like, “Oh, this is a real problem.” But we refused to give up on the idea that technology was just doing amazing things…

      Andrew: We weren’t gonna let a little evidence get in the way of all the <inaudible>, for God’s sake.

      Sonal: Yeah, no. No way. No dammit, we’re not letting that happen.

      Erik: But fortunately, we figured out a way out of it. And the way out of it is that even though technology is making the pie bigger, there’s no economic law that everyone’s going to benefit from it. It’s possible for some people to get left behind. Now, to be clear, that’s not what happened for most of the past 200 years. But the past 10, 20 years, there really have been more and more people being left behind. And so you could get stagnating median incomes, even as some people, maybe in the top 1%, got fabulously wealthy. And that helped us reconcile these different perspectives. And it led to a whole broader set of discussions about the way that organizations, and society, and business processes aren’t keeping up with these amazing technologies, and some of the dysfunctions that can create and some of the opportunities that can create.

      Sonal: So what are some of the big — well, I think we should break down the fundamental building blocks of a lot of the arguments that you make throughout your work. So let’s talk about networks. And one of the biggest questions I had reading your book was, is a network going to displace the firm in the future? We talk a lot about network effects in our business.

      Erik: So networks, sometimes economists call them demand side economies of scale — and it’s basically the idea that a product or service becomes more valuable the more other people that are using that product or service. A classic example is, you know, a telephone, or a fax machine, WhatsApp, Facebook. And you can have supply side economies of scale, just to distinguish that — that’s when the costs get lower as more people use it. And both of these things lead to the big companies winning.

      Sonal: And just for shorthand, we tend to describe supply side economies of scales as just economies of scale, the demand side economies of scale as network effect.

      Erik: That’s the more common way.

      Sonal: More generically.

      Erik: And we use both sets of terminology. It’s sometimes useful to talk about supply side and demand side, because a lot of the economics become more intuitive once you understand that there’s the demand side and the supply side, and they both can get better as you get bigger. And then to add a little more layer of subtlety to it, you can have traditional single-sided network effects, like other people using the same telephone, or you can have a two-sided network. And that’s what — really the platform revolution, a lot of that has been triggered by the growth of the so-called two-sided networks.

      And the idea there is that it’s not necessarily people using the same product as you, but it could be people on the other side using a different product. So, like, drivers and users are using slightly different apps. And me as a user, I don’t really benefit when more users are also, you know — I want more drivers. And the drivers want more users. So you care about the people on the other side of the network.

      Sonal: Except when you’re pulling, because then you do care.

      Erik: That’s right.

      Sonal: And that’s a case where you do want them on the same side.

      Erik: Exactly. And then to make it even more complicated, you can have two sided and one sided at the same time, you can have economies of scale. So you can layer them. You mentioned the word building block. Let’s start with these primitives. And then you can start combining them in different ways.

      Andrew: This really starts to turn into three-dimensional chess, because the right way to think about the app ecosystem in Apple is not any kind of one or two-sided network. It’s an n-sided network.

      Sonal: Multi-sided, yeah.

      Andrew: And lots of different groups of people who value things on the other side, but we don’t decide what the sides are, and we let the self-selection happen. And you just watch the vortex form around that ecosystem. And the only way to understand that is by doing what Erik just did — start with network effects, one-sided, two-sided, two goes to N, value goes to N.

      Sonal: Okay, that’s great. And then let’s probe on one big thing, which is we talk about network effects. But let’s quickly define complements in this, because that’s a term that’s frequently used. And I think it has a lot of misconceptions around it.

      Erik: Sure. One of the key economic building blocks that we talk about is complements. And a complement is a very simple concept. It’s the idea that one product is more valuable in the presence of another. So my left shoe is more valuable if I also have my right shoe.

      Sonal: Well, that’s an obvious example.

      Erik: Yeah, yeah. Software is more valuable with the right hardware. And so, complements can be physical, they can even be organizational. Well, so you may have a system that taps into the crowd that’s more valuable when you have a global internet that allows you to do that. So you can have organizational, or technical, or physical complements. And you can sell products that are complementary to each other.

      Sonal: The razor blade is the classic example.

      Erik: Yeah, razors and blades. And sometimes when you have products that are complementary to one another, it actually can be profitable to give one away to increase the demand for the other one. So people famously gave away razors to sell blades. And this can interact with the network effects and the scale economies. It’s not a good strategy if you don’t have those other things. One of the things that, you know, makes us tear our hair out is that, you know, when MBA students are like, “Oh, yeah, we’ll just give it away.” Like, where is the underlying strategy?

      Sonal: Oh, so if you’re just saying like, “I need to do freemium,” without really understanding the underlying strategy that we’re trying to accomplish.

      Erik: Exactly, disastrous.

      Andrew: And complements are weirdly subtle. And Erik just explained…

      Sonal: This is why I wanna ask about it. Because it’s a very nuanced concept.

      Andrew: Erik just explained them super clearly. The Econ 101 example that I always fall back on is hamburger meat and hamburger buns. And so, if the price of hamburger meat goes down, demand for buns is going to go up, even if the price of buns doesn’t change. That’s the key thing. The price of one good can stay the same and demand for it will go up. The complements are so tricky that they actually tripped up Steve Jobs really badly. This is not lore, this is fact. He did not want to open up the app store to any outside developers. He thought he had to maintain super tight control over that digital environment. And when the iPhone first released, it did not have any external apps on it. He fought boardroom battles for about a year with people who said, “No, you need to open this up.”

      Sonal: What made him cave?

      Andrew: Pressure from really smart people inside and outside the company, people on his board and executives at the company. What he didn’t fully realize is that if you open up the app store and you curate successfully, you have just opened the door to this massive number of complements, each one of which is going to nudge out demand for the iPhone. And even if each one only nudges that demand outward…

      Sonal: Like 99 cents worth.

      Andrew: Yeah.

      Erik: Oh, no, even less.

      Andrew: Even less.

      Erik: Just to be clear, we’re not talking about the literal money…

      Sonal: Yeah, I know. I know. Exactly.

      Erik: Yeah, we’re talking about the fact that it makes the…

      Sonal: It’s a relationship that makes the entire…

      Erik: It makes the phone…

      Sonal: It makes people want the phone. Remember the early days of the iPhone — I still don’t have an iPhone, I have an Android. But I still remember to this day, the first thing people would say I’m like, “I don’t really like the iPhone that much.” And they’re like, “Oh, it’s not about the phone. It’s all about the apps. It’s all about the apps.” That was the line all the time.

      Erik: Angry Birds.

      Andrew: Yeah. And the only way to understand the value of opening up that app store is to understand that you are unleashing this tidal wave of complementary goods that were priced at all different price points, including zero, which is awesome. So zero is a really great price. But the more fundamental thing, I think, is that it shifted out demand. It nudged demand upward for the other complementary good, the iPhone itself. And once you grok into that, then you say, “Oh, I got to find all kinds of different ways to do this and play three-dimensional chess with my platform.”

      Sonal: Is the corollary of all this that “closed” will never win then?

      Erik: No, it’s not nearly as simple as that. But it does show you that if you can leverage these complements, you can create not just a one-time win, but in a whole ecosystem, because Andy’s story turns into a virtuous cycle where the more demand for the iPhone…

      Sonal: Right, flywheel.

      Erik: Exactly. It’s a flywheel. So that can work very well. But it’s not like you always open up, or you always build complements.

      Sonal: Right. Because I was gonna say, a lot of the winners until now have been closed companies.

      Erik: Yeah, absolutely.

      Andrew: Yeah. And Apple was comparatively closed against Google and the Android ecosystem. One of the things we say is, there is not one right answer. There is not one recipe that you follow for success with machines, platforms, or crowd.

      Erik: There are principles.

      Frank: And for entrepreneurs who are listening, understanding complements, and the way the people who are creating these ecosystems that have complements is super important. So we’ve been talking about complements where the more apps in the app store, the more attractive an iPhone. So think about that when you’re thinking about development tools for these platforms. Xcode, Visual Studio are so important to Microsoft and Apple, because they’re creating these complements and therefore the desirability for their iPhone. That’s where they make all their money. So if you think, “Hey, I’m going to create a better development tool. I’m going to create a better Xcode.” Like, think again, because Apple is going to spend as much money as it needs to defend a complement universe.

      The future of the firm

      Sonal: Crushing you. The question that comes to mind for me is what this means for companies.

      Frank: So one thing that — conventional wisdom now is, we fund companies whose defensibility is a network effect. In other words, we’re in Lyft and Airbnb precisely because once you have all the hosts, you’re going to get all of the renters, right? And so, one thing to think about is, maybe in the future, even the firm that creates the network effect gets decentralized. Who needs a firm? Why don’t people just come together and we’ll create the right set of incentives for the network to behave? So you can imagine an eBay where there is no company. There’s just a network coming together with the right set of incentives.

      Andrew: That was how we wound up the book, is trying to grapple honestly with this question of in the universe that can be turbocharged by the fact that everyone’s got a device, that we’ve got this completely decentralized cryptocurrency system you could pay people with, that we’ve got these technologies of radical decentralization.

      Frank: Like the blockchain.

      Andrew: Like the blockchain.

      Frank: Like the blockchain. Public distributed ledger. Every transaction…

      Andrew: Where you could stick…

      Frank: …everybody’s <crosstalk>

      Andrew: …contracts and code into those things. You can do a lot of the stuff that we used to need a company for. The question gets teed up, are we still gonna have companies in the future? And as Erik and I started to think about all the stuff that we’d learned and tried to digest, our answer was an unequivocal yes. And the main reason for that is that ownership of a thing matters, simply because almost — well, every economist, I think, that we’ve talked to would agree that you can never write a complete contract that will specify exactly what everybody is going to do in all future states of the world.

      Sonal: Every possible contingency cannot be accounted for.

      Andrew: And the reason for a firm is it gets to make the decisions that are not contractually specified elsewhere. And it gets all the value that’s not apportioned elsewhere in the network.

      Erik: It starts with Ronald Coase…

      Sonal: Of course, the classic “Nature of the Firm,” 1937 or something.

      Andrew: ’37. He’s a hero. He was 9 years old when he wrote that. He was in his 20s or something…

      Sonal: Did you say 90 or 9?

      Andrew: No, he was in his 20s.

      Erik: Yeah, he was in his 20s, 26 I think he was. But then and then more recently, Oliver Hart, who was my thesis advisor, and Bengt Holmström, one of our other colleagues at MIT, elaborate on that, as Andy was saying, with this so-called incomplete contracts theory. One of the blinders that a lot of people, especially technologists have, is they say, “Hey, we can just write everything down under an engineering mindset. We’ll write a complete contract that covers all contingencies.” And the reality is, the world is just too complicated to cover every possible contingency. So when you own a car, you can sell that to someone else. And whoever owns the car gets to have all of what are called the residual rights of control, everything that’s not specified in the contract. You want to change the color of it, that’s what ownership means. And ultimately, you take that to the level of the firm. A firm is an aggregator of a bunch of assets and owns certain things. And that means, that gives them a certain power, that gives them certain incentives of how those objects are used.

      Andrew: As Erik and I were trying to reason our way through this and convince ourselves to one view of the world here, this amazing real-life experiment happened, which was the Dow.

      Sonal: Yeah. And let’s do a quick terminology thing. When you say the Dow, you mean the corporation that was formed, but that’s very different than a DAO which is a decentralized autonomous organization or decentralized autonomous corporation. This is the Dow, the entity.

      Andrew: This is the thing called the Dow.

      Sonal: The proper noun, not the generic noun. Yes.

      Andrew: The Dow, which was intended to be a completely owner-free, completely decentralized organization along the lines that you just described. And it got hacked, and somebody found out how to treat it like an ATM essentially. So, to the extent there was a group of people, kind of, behind it, they collectively freaked out and thought about what to do. And then they made this fairly autocratic decision — looks a lot like an ownership decision for me, to reset the clock on the entire Dow.

      Erik: They became de facto owners, they asserted those rights in a way.

      Sonal: That’s right.

      Andrew: A de novo. They said, “Okay, we’re gonna do this. And if enough of you go along with this, then this is what’s going to happen.” It was extraordinary for a very decentralized organization, it was kind of heavy.

      Sonal: I mean, I love you’re saying something counterintuitive, which is a firm is not going to go away. It’s gonna actually look the same as it does now, then. But when we talk about the transaction cost of all this coordination, and why you need management — or even you have this incomplete contract theory, and people — you can’t predict every contingency. What if we have an algorithmic AI who’s able to then account for every one of those contingencies versus — we’re basing our theories right now on what we know already. We don’t know how it’s gonna play out in the future.

      Andrew: Amen.

      Erik: Well, we’ll never say never. And yeah, if there’s an AI that has magical properties that we can imagine, you know, all bets are off, of course. But we’re talking about a world right now, where the blockchain and related technologies are allowing radical decentralization of lots of types of decisions. And that’s really important. It’s changing, creating a lot of new opportunities, but it doesn’t change everything. And there are still some core things like this concept of incomplete contracts. Anything that’s not explicit that you can’t write down, maybe you can’t anticipate, and maybe the current AIs can’t anticipate, then those are the residual and that’s where ownership actually…

      Andrew: That leads to something like “company” being an enduring part of the economic landscape.

      Sonal: I mean, I would even make it more basic, which is, it’s human nature that people — at the end of the day, systems of networks that are online, or in a company, or any other form, are made up of people, and people are fallible and are emotional.

      Andrew: And fractious, right?

      Sonal: Yes, they wanna fight.

      Andrew: If we look at the breakdown in the Bitcoin community and the civil war going on there. Okay, one reason you have management is to say, “Gang, we’re going to go this way and not that way.” And disagree and then commit, as opposed to disagree and then disagree.

      Erik: And we all have bounded rationality. Friedrich Hayek called it — was the fatal conceit, the idea that we could plan everything in excruciating detail. The world is far too complicated for any one person or any one group of people to do that. There’s even a, kind of, a Red Queen phenomenon, that the more sophisticated you are, the more sophisticated your competitors are, your customers are, your suppliers are.

      Sonal: Why is it called the Red Queen phenomenon?

      Erik: Oh, so Alice was…

      Sonal: From Victoria Aveyard’s novel, or…

      Erik: No, from “Alice in Wonderland.”

      Sonal: Oh, from “Alice in Wonderland.” Of course.

      Erik: You have to run faster and faster…

      Andrew: Faster and faster just to keep up.

      Erik: …just to stay in place.

      Sonal: Got you.

      Erik: So if you get more sophisticated, all those other parties are getting more sophisticated too. You’re not going to be able to completely anticipate what they all do, because they’ll be even more clever.

      Andrew: But think about how crazy this is. Hayek brought up the term “the fatal conceit” to demolish this idea that we could centrally plan an economy. And at the time, when a lot of intellectuals in the West were excited about Soviet-style central planning, Hayek wrote one paper and just demolished it. There’s an almost 180-degree reverse — perhaps “fatal conceit” going on — among the fans of radical decentralization as opposed to radical centralization.

      Sonal: Right. So you’re saying the same phenomena is at play, just in a different direction. But I wanted to add something, too, because I was gonna say, there’s now some claims out there that the power of simulation has gotten so good that we might be able to actually move to that fatal conceit of being able to centrally plan an economy, because of all these data and machine learning, you know, sort of, signals and whatnot.

      Erik: So, Alan Greenspan, of all people, I asked him about computers and the ability to simulate the economy. And he was a chairman of the Federal Reserve, you know, set interest rates and everything. And he said, “Well, yeah, we can understand a lot, lot better. But all the companies are reacting that much faster as well.” And so it’s exactly this Red Queen phenomenon, that however much the Federal Reserve advanced, each company advanced, all the other guys are doing the same thing. If you could freeze the rest of the world, and you were the only party that had access to cloud computing and Moore’s Law, etc. Yeah, maybe you could stay 1 to 10 steps ahead of them. But that’s not the way the world works.

      Frank: There’s a great story from the early days of AI on this fatal conceit idea, which was in the late ’80s, Japan tried to organize their entire industrial policy around creating artificial intelligence.

      Erik: Fifth generation.

      Frank: The fifth generation. Supercomputer.

      Sonal: Like what’s happening in China right now.

      Frank: Built around expert systems, optimized all the way down in the silicon. So you can imagine, silicon optimized for Lisp, right, so that we can build apps. <crosstalk> And it was a complete failure, precisely to this idea of — you actually can’t plan anything, right? What happened out of the ’80s was more the rise of client server computing and Microsoft Windows. Nobody anticipated that.

      Andrew: And the idea that we’re out of that world because of Moore’s Law, because we have much more computational power now, I find that ludicrous.

      Sonal: Well, tell me why? If we have this accelerating, growing, fast-happening thing — and I don’t want to make it a crutch to say, like, we can’t predict the future, dot-dot-dot, blah, blah, blah. We already know that. But why not? A lot of things that were tried before didn’t work because it was the wrong time. Why wouldn’t that be possible now? Like, can’t simulation work there?

      Frank: Yeah. I mean, speaking as an investor, you know, who’s trying to predict the future and often gets it wrong.

      Sonal: As you should.

      Frank: You know, it’s hard to imagine a better system than the one we have, which is, let’s spend a little money and run a ton of experiments on businesses to figure out what people want. Because until you have it in the world, you’re not sure what the people will want.

      Andrew: And that’s not called simulation in the face of massive computational power. That’s called entrepreneurship and capitalism. It’s a very different approach.

      Sonal: I agree with you guys. I find that.

      Erik: So, if anything, the data is going the opposite direction.

      Sonal: Which is?

      Erik: We’re seeing less planning and predicting, less five-year plans, we’re gonna do this, and a lot more experimenting, testing, fail fast. That seems to be a model that works a lot better.

      Sonal: But the other thing I was gonna say is, like, I look at countries like China and their incredibly coordinated efforts. And while I agree that past central industrial planning efforts have failed, for various reasons, I don’t know, I think there might be something to it this time. I just want to make sure you guys really disillusion me of that. Help me let it go.

      Andrew: And our colleagues, Daron Acemoglu and James Robinson wrote this amazing book called “Why Nations Fail.” And their answer was really straightforward. Nations fail because they have extractive…

      Erik: Extractive institutions.

      Andrew: Extractive institutions, where an elite grabs power and they just suck up the value of more.

      Sonal: Arguably, that’s why companies fail too. <crosstalk>

      Andrew: Exactly. And they make sure that their descendants…

      Erik: Yes, that’s a good analogy. You should write the next book.

      Andrew: And they hand down power to their descendants, and they just make sure that they pervert the rules of the game to benefit themselves. That’s as opposed to inclusive institutions, where you have an honest shot of making the most of your human capital. Now, which one is China? They took big steps in the direction of inclusion by opening up to a market economy. Would we call that authoritarian state, one of actually inclusive institutions? I would not.

      Sonal: I think that’s the legitimate thing to say. Okay, so just going back to this idea of extractive institutions. So, I do think it’s interesting that there are now networks that are coming up that are letting people participate differently as owners…

      Erik: For sure.

      Sonal: …in different ways. And that is where I think this topic of ICOs and token launches is really interesting.

      Erik: Part of the power, as Hayek would have said, is that you decentralize some of the local knowledge. They have information that nobody else has. And if you could…

      Sonal: That’s right, or resources, like if it’s a computing power…

      Erik: Yeah, they have skills. Exactly. If you can move the decision rights to where that knowledge is, you’re going to be better off. And one of the great things that technology has allowed us to do is move around decision rights, move around ownership. So hopefully, if you do it right, you get a better match between the incentives and the decision rights.

      The power of crowds

      Andrew: The entire third section of our book is about this rebalancing necessary between the core institutions of a company, and the crowd available over the internet now. How much more room there’s very likely ahead of us, with crowdfunding, with crowdsourcing, with different ways to tap into what people can do to give them an ownership stake, to get them bought in and pointing the right direction. Have we scratched the surface of that?

      Erik: Let’s talk a little bit about Joy’s law, that no matter what company you work for, most of the smart people in the world work for somebody else. It used to be limited what you could do about that, because there’s only so far you can communicate. But now for the first time in history, a majority of the world’s people are connected with a digital network. So they can access all the world’s knowledge. And part of it isn’t necessarily that they’re smarter out there, part of it just comes from the raw variety, the diversity, the variance. Within a company, you tend to have people who are like-minded, they’ve trained the same way. That’s who they get hired. And maybe the way to solve a problem is with an entirely different approach. And that may be somebody from a different culture, a different way of looking at the world.

      And you’re very unlikely to have that diversity inside of a company. It works against it. But if you can find a way to tap into it. One of our colleagues, Karim Lakhani is now at Harvard Business School, he was a Ph.D student at MIT, has done just case study after case study of examples where tapping into the crowd blew away what companies were able to do internally.

      Andrew: He worked with the National Institutes of Health to try to improve the speed and accuracy of sequencing human white blood cell genomes, which are really complicated but important to sequence. The National Institutes of Health, which I would call the core of the medical establishment.

      Erik: Core in the sense of core versus crowd.

      Andrew: They had an algorithm that could do a run in about four hours with about 70% accuracy. There was a faculty member at Harvard Med School who made a big improvement to that algorithm. He developed one that got them up to about 75% accuracy. Karim then worked with the NIH and Topcoder to make this an algorithmic challenge and open up to the crowd. And the best solutions got down to about 10 seconds and about 80% accuracy.

      Erik: From 4 hours to 10 seconds.

      Andrew: So we called up Karim and he goes, “About average. When I run a crowdsourcing tournament, this is the magnitude of improvement I expect to see.” The last part of that story that continues to blow us away is that they interviewed the best performers who submitted the top-performing algorithms. None of them had a life sciences background. There was not a geneticist…

      Sonal: Oh, that’s the best part of the story.

      Andrew: …there was not a biologist among them.

      Sonal: So crowds and prediction markets are similar. What’s the difference?

      Andrew: I would say a prediction market is one way to harness crowd wisdom. Markets do a really good job, overall, on aggregating knowledge.

      Erik: Markets tap into the crowd. Google taps into the crowd because their search algorithm basically exploits the link structure that all of us contribute whenever we make pages. There are lots of ways of tapping into the crowd, but being clever about how to reach them, motivate them, aggregate them — still a lot of work to be done on that.

      Frank: Let’s talk about the nature of work. Because I think what people do in that firm, either inside or outside, probably changes a lot. So, we have this idea that human decision-making is sort of fundamentally flawed in that, like, there’s biases that you bring to your decision-making that you don’t even understand. So when you’re thinking through, you’re still going to make the same mistake because you don’t understand that you have that bias.

      Andrew: After all, walking you through your decision-making process is your brain that came off that flawed decision-making process in the first place. It’s not going to catch its own mistakes typically.

      Frank: Right. So it’s a permanent blind spot. And by contrast, you would sort of assume that a machine learning algorithm, trained with a carefully selected broad set of datasets, will have a decision-making efficiency or effectiveness better than, you know, flawed humans. So if that’s the case, what do people in firms do? Like, how do you prepare for this world, where there’s going to be machine learning algorithms that can, in general, make pretty good decisions. And then there’s this idea that, like, maybe the talent is better outside your company than inside your company. So what should you do? Should you join a company?

      Erik: It’s just breathtaking what it can do. But it is far, far from being AI complete, being able to do everything that humans can do. There’s a certain class of problems that it’s kicking butt on, but that’s a tiny sliver of what human decision-making is. Even just defining what the problem is, exactly what needs to be done, that’s half the battle. But you need humans to do that. There’s a quote that we had from the book from Picasso, “Computers are useless. All they do is give you answers.”

      Sonal: I was a little shocked Picasso was alive when computers were…

      Andrew: He actually said that. We went and wholly investigated that one. He said that.

      Sonal: I know. I just never associate Picasso and computers. It’s amazing.

      Erik: Well, he’s a brilliant guy in a lot of different ways. And obviously, he didn’t know much about the latest neural network systems. But his understanding was spot on, that simply giving the answer isn’t necessarily the most interesting or important part of solving problems.

      Sonal: Kevin Kelly actually makes this argument in “The Inevitable.” We had him on the podcast, that the number one job of the future for humans that humans preserve — and this is I think what you’re getting at — is that we ask the questions and computers answer. But I have to say, I actually disagree with that a little bit. Because I’m seeing a new class of generative AI that makes me wonder if they’re going to be asking you questions that make us want to answer differently. I mean, there’s all kinds of interesting things.

      Erik: Our brains are made of atoms and so are computers. You know, I’m not going to say that there’s some things that they just can never touch.

      Andrew: But I agree, which is that on average, our wetware is amazing. But it’s got a host of bugs, and biases, and glitches in it, that machine learning systems, and properly-configured algorithms in general do not have. So, if you could only pick one of those two entities to help you — the good news is, that’s a false choice. We don’t have to make that choice. And I think the art going forward is being more clear about, “What are we actually good at?” versus what the machines are actually good at. The happy news is that they have very different failure modes.

      Erik: Yeah. And I think that’s exactly the key point. It’s a matter of how we can leverage each of them. Because machines have biases as well.

      Andrew: Yeah, algorithms are biased by definition.

      Erik: It’s not just [that] somebody designed them, but also the training data that they get. I mean, if you decide to give loans based on all the loans that have been approved or rejected in the past, that could have some biases built into it. And some of these neural nets could have billions of connections. Getting it to sort out how exactly — it’s not gonna be one of those — says, “Okay, discriminate against women.” But there may be some very subtle interactions that are hard to anticipate or explain. That said, at least the machines can be tested and improved. And it’s often easier to do that than it is with humans.

      Andrew: We are really resistant to having our wetware tweaked. We really just don’t like to be told that we’re glitchy, and here’s the fix and just go do that, no. There’s a concept…

      Sonal: That’s the story of most marriages.

      Andrew: Yeah, most marriages and most everything, right? It’s really, really hard to do. There’s a concept from linguistics that I find incredibly helpful for helping understand what I think some of the most durable human advantages in a world full of machines will be. And it’s a concept called the intuition of the native speaker. And what they mean by that is, if I look at any English language sentence, I can immediately tell if it’s grammatically perfect or not.

      Erik: You just hear it in your head.

      Andrew: Yeah. We are the native speakers of the human-created world. Computers are doing this as their second language. I believe we have a massive advantage. We are the native speakers about this reality around us.

      Erik: Rather than trying to build a system that does everything from soup to nuts, you get some kind of a division of labor. Sebastian Thrun described a system to us recently that was just fascinating. He’s at Udacity. And a lot of…

      Sonal: Another a16z company.

      Erik: Yeah.

      Andrew: Rock on.

      Erik: All right.

      Sonal: We make good investments, hey.

      Andrew: Listeners at home, we don’t have a list of a16z companies that we’re ticking off.

      Sonal: Yeah. I was gonna say, this is all natural, organic. Nobody’s planned a thing, I was just gonna say.

      Erik: We do have a list of cool companies, you know, which seem to overlap for some reason. But, you know, Sebastian described how they get incoming traffic in their chat rooms of people asking about their offerings. They decided, “Let’s take this data and we’ll see which of these conversations lead to sales, which ones don’t lead to sales, and label them that way. And then train a neural net about which replies were successful.” And then what they took with those replies, they didn’t try to have a standalone chatbot that then talked to customers. Instead, they had the human salespeople keep interacting. But when they saw one of these more common error modes, they would gently prompt the not-so-good salesperson, you know, “Maybe you want to give them this set of answers or this other set of responses.”

      Sonal: So it’s kind of getting their argumentation idea.

      Erik: It’s absolutely argumentation. Because there’s a long tail of other questions that the bot had no clue what they were about. So it could help with the most common sets of queries. And this is, I think, a pattern that you see lots and lots. You see it among radiologists. You combine the two and you end up having fewer false positives and fewer false negatives.

      Frank: Yeah, I love this idea of sort of machines and humans working together. And I think it’s only a matter of time before we walk into a doctor’s office or a lawyer’s office, where that isn’t the fundamental interaction, and we’ll just be horrified like, “Where’s your AI companion? Why are you trying to do this yourself with your biases?”

      Sonal: Oh, that’s fascinating.

      Andrew: I couldn’t agree more. Why on earth would I expect my GP, who’s a really good doctor, to be on top of the accumulated mass of human medical knowledge and keeping up to date with the latest developments in all the fields that might relate to what I walk in the door with? That’s an absurd request on a human being. Now, I want that person to be well trained. Even more, I want them to be able to empathize with me, and get me to go along with the course of treatment and get me to buy-in to what’s going on. Because that AI in the background that’s got access to my test and my lab results, again, assessed jaundice in my skin and, you know, how white the sclera of my eyes are, that’s going to be the diagnostic expert in the not too distant future at all.

      Sonal: That everybody wants, right?

      Frank: That’s exactly right. And I want AI not in the backroom, I want it in the room with me when I’m doing the conversation with the doctor.

      Sonal: A seat at the table.

      Andrew: You’re right.

      Erik: Well, with a seat at the table. It’s a theme that comes up again and again. We talk about mind and machine, product and platform, core and crowd. And we don’t want to give people the mistaken idea that you just cross off the first words of each of those lists and only do the second one.

      Andrew: The mantra that I’ve learned is that tech progress rewrites the business playbook. And what the two of us believe is that the way the playbook is being rewritten these days is in favor of machines, platforms, and crowds. So the balance needs to shift more in those directions.

      Sonal: So the playbook is in favor of machine, platform, and crowd.

      Erik: As opposed to…

      Andrew: Mind, product, and core.

      Erik: Right. So each of them is really a dichotomy. And the most successful systems are rarely all one or all the other.

      Andrew: That’s right.

      Applications for today’s businesses

      Sonal: A couple of threads that we didn’t get to pull. One question I had when we were talking about not all the talent is inside your company. And, you know, a lot of people talk about open innovation as a way to kind of get around that, like, open source communities, etc. What does that mean for business concretely? What does that mean for core, in the way that you’re defining core, and deploying the power of the crowd? Like, does a business whose main strategy is their core business, does that mean that all their innovation is now outsourced to the crowd? Or is it the other direction? What’s the ideal framework?

      Andrew: I think way too many, even successful companies today are overweighting their core. They’re probably spending too much of their total budget on it, way too much of their managerial bandwidth on it. And at the risk of being a little bit cute, I think a core capability for most organizations going forward is going to be interfacing with the crowd, harnessing its energy and its abilities, and then finding out how to bring that back into the organization without setting off all kinds of antibodies, and resistance, and nonsense.

      Erik: It’s part of the same lesson we learn from the mind machine trade off — is that defining the problem is important. Whether you define it for the machine, or whether you define it for the crowd, understanding what the problem is you’re really trying to solve. If you can define it well enough, then these contests work great. The contests don’t work great if you just say, “Hey, guys, you know, tell us stuff.” You give them a really precise…

      Sonal: Which is what people used to do with the olden days. Remember when companies used to do these crowdsource and innovation boxes.

      Erik: Yeah, and it never worked.

      Sonal: Yeah, it never worked for a reason. So then that begs another question, though, for me, which is, if you take the innovation from the crowd, and you said earlier that there’s this escalating effect where everyone has access to the same tools, and they’re all catching up really fast with each other, and you can’t — it’s always the Red Queen. You have to run faster than everybody else. But if everyone has access to the same crowd, how does the company get advantage in this space?

      Andrew: Then, honestly, it’s a matter of where your leadership throws its attention, how firmly you believe in these new kinds of energies out there. Not how willing you are to open the checkbook and spend money on technology, but how willing you are, forgive me, to open up your brains and rethink your business model in the face of this craziness.

      Erik: Who can use these tools more effectively? Just like who can use the cloud more effectively? I mean, it’s a matter — it’s like what it always is. It’s just a set of weapons out there. And some people have a better strategy. Some people have better techniques.

      Andrew: The companies that failed during the transition from steam power over to electric power, almost none of them failed because they refused to invest in electricity. That was not the failure mode. The failure mode was, they refused to rethink what a factory could be.

      Sonal: And how to really absorb into the core of their business, yeah.

      Andrew: And they refused to take seriously the idea of an overhead crane, or an assembly line, or a conveyor belt.

      Sonal: Yeah. I’m just thinking about the statistics. When you said this thing about this antibody that organizations naturally have, which is essentially — they just immediately reject this not-invented-here syndrome, basically, a disease.

      Andrew: Yeah. Look, and those antibodies are the best news possible for your industry.

      Sonal: Research has shown over, and over, and over again, that it is practically impossible for big companies to absorb startups successfully unless they keep them isolated. And one of the questions I have is — the next follow up, basically, of what happens when you leverage this crowd. How do you then really bring them into the company so that you don’t have these antibodies? Do you have any concrete advice?

      Andrew: I would look to do that in some of the most forward-thinking parts of the organization. As Erik said, in parts of the organization where the problem can be most clearly defined, and where you’ve got people at the helm of that part of the organization who are willing to take the innovation, the algorithm, whatever that the crowd comes up with, and slot that into the work of the organization.

      Erik: There’s a role for the core to be able to define that.

      Frank: In our world, a perfect example of the core leveraging the crowd is the classic enterprise software company. So, in the old days, basically, you wrote software, it was all proprietary. You won Gartner Magic Quadrant, then you sent your Rolex-wearing direct salesperson to go sell it to someone. The new enterprise company is, “Let me create an open source project. Let me get a lot of contributors. Let me get contributors to get downloads. And that’s my path to market.”

      Sonal: Right. The open source becomes, like, the…

      Frank: And the core needs to be there because they got — what’s the project? And what problem are we trying to solve? But the crowd comes into it to basically lend legitimacy, and support, and enthusiasm for the project.

      Erik: So if you can be that scarce complement to the abundant crowd, you can create a lot of value. Then you become the linchpin that is capturing a lot of the value as well as creating it. Ultimately, we are an economy of creative destruction. And one of the strengths of the United States and other dynamic economies is that we have this constant turnover. And one of the things that discourages us is that there’s actually fewer startups, less innovation, fewer young firms in America today than there were 10 or 20 years ago.

      Sonal: Oh, yeah. We talk about this phenomenon. That worries us too.

      Erik: Absolutely. We are all for trying to make the bigger companies more nimble, understand this.

      Andrew: Amen.

      Erik: But the bigger way that the economy innovates is by having this innovative set of new startups that rise and adopt some of the new technologies. You got to have both. And we’d like to see progress on both dimensions.

      Sonal: That’s great. Thank you for joining the “a16z Podcast.”

      Andrew: Thanks for having us on. This is fantastic.

      Erik: It’s a real pleasure.

      • Erik Brynjolfsson

      • Andrew McAfee

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Monetizing Open Source (Or, All Enterprise Software)

      Sonal Chokshi, James Watters, and Martin Casado

      Here’s what we know about open source: Developers are the new buyers. Community matters. And there will never be another Red Hat (i.e., a successful “open core” business model … nor do we necessarily think there should be).

      Yet open source is real, and it’s here to stay. So how then do companies build a viable business model on top of open source? And not only make money, but become a huge business, like the IBMs, Microsofts, Oracles, and SAPs of the world? The answer, argues James Watters, has more to do with good software strategy and smart enterprise sales/procurement tactics (including design and a service-like experience) than with open source per se — from riding a huge trend or architectural shift, to being less transactional and more an extension of your customer’s team.

      Watters, who is the SVP of Product at Pivotal (part of VMWare and therefore also Dell-EMC), is a veteran of monetizing open source — from OpenSolaris (at Sun Microsystems) to Springsource (acquired by VMWare) to Pivotal Cloud Foundry — with plenty of failures, and successes, along the way. He shares those lessons learned in this episode of the a16z Podcast with Sonal Chokshi and general partner Martin Casado (who was co-founder and CTO of Nicira, later part of VMWare before joining Andreessen Horowitz). These lessons matter, especially as open source has become more of a requirement — and how large enterprises bet on big new trends.

      Show Notes

      • General strategies for open source companies [0:44] and differentiating in the marketplace [9:26]
      • Dealing with competition from large firms [13:34]
      • Why open source is growing [18:20] and a discussion of Red Hat, OpenSolaris, and other case studies [21:26]
      • Advice for entrepreneurs on open vs. closed source [26:48]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. We’ve talked quite a bit about open source in the podcast already, from the topics of open versus closed, to managing community and identity, to selling to developers. And a few years ago, partner Peter Levine put out a piece arguing why there would never be another Red Hat, which is one of the only open core business models to survive. But given the current and coming wave of companies built on top of open source, the tricky question left to discuss is, how do they make money? And joining us to have that conversation, we have James Watters, who’s the SVP of product at Pivotal, a cloud platform company that runs software in multiple clouds, and they’re part of VMware. And then also moderating this podcast, we have general partner Martin Casado, who himself came out of VMware, which had acquired the company he co-founded and was CTO at previously, Nicira. And he is the first voice you’ll hear.

      Open source strategies

      Martin: One of the paradoxes in this entire space is, there’s been a ton of money that’s been invested in open source, but almost no examples of successful companies built around it. Silicon Valley had a spidey sense that there was an opportunity there, like, nobody pulled it off. Yet now, there’s a number of examples of open source companies doing very well. James is one of the few people on the planet that’s cracked that, where they’ve figured out how to monetize and build a big business out of open source. And just to give a sense of how real this is, Pivotal Cloud Foundry went from 0 to $270 million in license, not support.

      Sonal: I mean, what’s that mean, in license?

      Martin: In license and in software sales.

      Sonal: Oh, so it’s not professional services…

      Martin: It’s not professional services, exactly.

      Sonal: …because that’s the stuff that has like — that’s the thing we talk about all the time, in terms of building businesses, is that you don’t necessarily wanna rely on professional services because it doesn’t give you a lot of margin on your business.

      Martin: Exactly. So this is, like, legitimate software sales.

      Sonal: So, James, tell us how you went from 0 to 270 — million, we’re talking about, not seconds. How did you do it?

      James: Yeah. And I think it’s fair to describe, kind of, my maturation and my thinking about this through failures, too. I worked on OpenSolaris at Sun, and then, for a while, at VMware, we were looking at how do you monetize SpringSource in and of itself. SpringSource is, like, the most popular Java programming framework in the world, by VMware 4, pretty famous, money over $400 million as an open source project. I had worked on that, you know, both of those projects, and had gotten my bumps and bruises along the way. And so I had some very particular opinions coming into doing the third round of PCF, Pivotal Cloud Foundry. I think there’s, kind of, two dimensions. One is the basics, which is, if you look at IBM and Oracle, Microsoft, SAP, etc., what they get right in this very basic thing is they understand how to cater to enterprises.

      Sonal: Yeah.

      James: And I think the number one temptation that most open source companies fall into is the first thing they do is they cater to their users.

      Sonal: And by users, you mean the developers?

      James: The developers that, you know, fork it on GitHub, start on GitHub. And the misassociation, as a first thing between the people that use it and then the people they need to cater the selling to, is kind of what you might call the first false horizon of open source monetization.

      Sonal: Interesting.

      James: What I mean by cater to is, like — I think this is the number one thing that I see open source companies maybe get wrong — is those are the people that show up and talk to you, those are the people that you’re interacting with. When you’re creating an open source community, you have to, first, get these unpaid users excited. But ultimately, IBM and Oracle are not out there mining their mailing lists of end users to go do deals.

      Sonal: Yeah. How do you sort of straddle this? Because theoretically, you should be able to do both. But isn’t sort of the mantra, I should say it the Indian way, of open source, that you have to take care of your open source community and then also figure out how to become a business? Like, how does a company that’s trying to become a real big company straddle that?

      James: It’s super important, right? I’m not saying that that’s not. What I’m saying is that your commercial strategy and your community strategy are not the same thing.

      Sonal: That’s a great point.

      James: It’s tempting when you put so much heart and soul effort into building that proof of validation around the community first, to say, “Oh, well, then, I’ll go upsell them.” And I remember meeting MongoDB when they were early in their rise, and they were selling, you know, a $4,000 support contract to thousands of users with a lot of expense. And when we built PCF, I was like, “I don’t think that’s the right way for us to go be big.” Just because you’re open source doesn’t mean you only sell to your open source users.

      Sonal: So what did you do in getting across this first horizon?

      James: The next thing you have to get right is you’ve got to get a major trend that affects enterprise buyers. So if you go back to enterprise as being the source of the money, you know, Oracle didn’t magically just exist one day. They caught the relational database market at the right time and built a franchise. And that relational database market really changed how enterprises were building applications, what they could do on applications. And in the same way, we’ve tried to catch the microservices trend as the major enterprise change that’s happening. That’s the kind of change that you need on your, you know, commercial strategy to generate CIO interest and enterprise purchases. It can’t just be a tool that someone’s potentially using [at] lower levels.

      Martin: There’s been, [for a] long time, this thought that open source is primarily a commoditizing force. There’s an existing market, say, Unix. It’s an existing market with an existing buyer. Then you create an open-source version, which is differentiated by being open on open source, and then you go commoditizing that existing market. And you’re saying something quite a bit different. You’re actually saying that you can enter a new market using a shift and sell into that. So, I guess two questions. One, do you believe in the commoditization? Is that still a business plan? And then the second one is, like, is there any difference in open source to identifying these shifts, or is it just like if you’re doing a closed-source product?

      James: Let’s look at history a little bit. I think Zen and OpenStack both felt like they could go commoditize VMware, right? They both said, “We’re gonna go commoditize VMware.” The last time I checked, Oracle and IBM did not get to be the size they are by having commoditized a previous generation of suppliers. So if you just look at the record — and this is what I call, like, the false horizon of open source strategy — you might be tempted that your only play is to be a commoditization play. But history tells us that the largest software companies in the world that get into the hundreds of billion in valuation, that’s not what they did. They caught megatrends.

      Sonal: They built something new. They didn’t just commoditize something old.

      James: They did. And I know that that sounds, like, obvious…

      Sonal: I’m glad you’re pushing on it.

      James: I’ve learned, over the years, that open source is now both a necessary part of almost any major software company strategy, because it’s a buy-in criteria that, you know, enterprises have for major new initiatives. But — that I think the open source got a little bit of a false start by only being a low-cost commoditization as a business model.

      Martin: From my conversations, it’s almost somewhat of a contrarian view. Because traditionally, you’re like, “Open source projects chase after, basically, the sales dollars that have matured to market,” like MySQL, you know, like Android, like Linux, like JBoss. Every one of these you can point to a closed source incumbent that they were chasing after. So it’s nice to hear that you believe that open source is not somehow relegated to this commoditization. Because I agree with you, that basically limits the upsides you’re gonna get.

      James: It does, because it limits your business model. It doesn’t let you invest. We have a $100 million a year R&D budget. When you’re chasing a new big strategic trend, and you’re getting the kind of checks that we are and the trust we are, you can build a big R&D team. If you’re chasing commoditization… 

      Martin: Well, I mean, you are trapped to a fraction of the market you’re chasing after, by definition, right? Like, if it wasn’t a fraction of it, you wouldn’t be commoditizing it, and it’s very unlikely you’re gonna get it 100%. And so you necessarily, you know, have a ceiling over you. Well, it sounds to me that your recommendation is a lot like closed-source software sale. However, is it more than just, kind of, software sales, or is it just that, like, the industry is ready for open source now when it wasn’t before? Like, what has happened in the last two years to make this viable? Or is there more to the puzzle that you haven’t talked about?

      James: And so the basics are strategy and segments. If your strategy and segment looks wildly different than every other big software company that’s existed before, that’s probably a pause. And then the second thing is, I think that cloud has started to affect open source, and we’ve taken a model of continuous delivery of our software, inclusive of cloud API. We don’t do what I call shipping you the tarball and the support contract and say, “Good luck.” We actually automate the continuous deployment and update of our software. And so we have an additional point of leverage other than just support, which is that, if anything doesn’t work in that large-scale update of thousands of nodes, you just blame us and call us. And that’s a different vector. That’s almost like a cloud vector in terms of value-add of software packaging.

      Martin: That’s exactly right. We hear this all the time, which is, more and more, the customer wants to consume things as a service.

      James: Right.

      Martin: And this is often conflated with whether it’s deployed off-prem or on-prem. Like, to me, this is a total conflation, right? Like, whether or not it’s a service does not mean it’s necessarily deployed off-prem, right? These are two things. So one decision, off-prem or on-prem. Like, that’s actually a decision often, like, bound by regulatory compliance and security, etc. But there’s an entirely different decision, is — my consumption model as a service. That could be, I pay for it as a service, but also could be, somebody else basically manages it, does the update, manages the lifecycle, I don’t deal with the tarballs. That’s very much in line with what we’re seeing across the industry too. And what makes this discussion particularly relevant to open source is that it seems that once you’re talking about something as a service, questions around open source kinda diminish, because they’re not actually dealing with the code itself, they’re dealing with the service.

      James: Big success stories sometimes have contrarian bets in them. And I would distil our two contrarian bets to, we did not go for the commoditization play, we went for the sea change and app design play.

      Martin: That’s right.

      James: And then we also tried [our] best to deliver a service-like experience even with software.

      Differentiating in the marketplace

      Sonal: Are there any challenges, though, in sort of bringing a polish to open source work? Because when I think of traditional software companies, they have baked in design for user — like, really client-facing versus developer-facing. So how do you sort of navigate that part of that, as building a real business on top of open source, if you don’t have that experience natively?

      James: One of the decisions we made is we made our UIs closed source. So everything about the infrastructure of the platform is completely open source. And we chose to make the UI in that last mile of experience built off the APIs closed source.

      Sonal: Yep. Okay.

      James: So I do think there is some room to differentiate there, and you know, when you go to monetize, you’re going to need some small checkboxes. I’ve learned a lot, you know, growing this, about the importance of packaging for procurement. And one of the things I’ve observed is that procurement is exceptionally good at pricing a certain kind of thing, which is labor per hour. So, I’ve seen very large software contract orders, and procurement will actually pick on the labor per hour, because that has a nearer comparable. And so when you start to think about procurements looking to compare things to exact replicas to price it, having a little bit of, you know, closed-source UI or things that are maybe even immaterial to the product but are still there to say, “Oh, well, this is different,” is somewhat important.

      Sonal: Interesting.

      Martin: That’s really interesting.

      James: And thinking about dynamics of navigating the politics of procurement are important.

      Sonal: What are some of the other dynamics of navigating the policy of procurement, your lessons learned that you can share with us here?

      James: I mean, you guys are asking for the goods now.

      Sonal: Yeah, we are. We want the secret sauce. Give it to us for free. Share it with our listeners.

      James: I mean, I think that is — generally, in an enterprise, when you get to procurement, somebody wants you to win, and you’re actually, then, in a political process to get through the last mile. Like, they’re kinda, like, the guardians there.

      Sonal: You mean the internal champion type of person?

      James: Internal champion wants you to win. Like, if you’re in procurement, they want you to win, and they wanna work with you. And this is why one of my other rules about open-source monetization is, try to avoid dogpiles. So a dogpile is a little bit, like, everyone’s doing X project, we’ve got to distro of X project. Think about the dynamics when you go into procurement then, and 20 people show up with an offer for X project. Like, no matter how much your champion loves you, the procurement officer is gonna have 10 comparables to compare you against. If you look at MongoDB, for instance, one of the things that protected them was that they are the only supplier of Mongo. So while they might not have had the high-end strategy right when they started, they at least were the sole supplier. So they could still somewhat dictate prices. If you look at OpenStack distros, I don’t know that anyone made it out alive [out] of that gunfight. 

      Martin: This is such an important point. And I think it’s actually worth restating, which is, if you’ve originated a project and you’re bringing that to market, you should retain the ability to be the sole supplier, if only to set pricing and brand awareness in the market. Companies live or die by this type of thing. Another thing that I like to hear, if this is true for you, having spent a lot of time dealing with procurement — that more than anything else dictates the life of enterprise sales. Before going into procurement, I always expect procurement to need their pound of flesh. Like, this is how these guys are comped often. It’s like, “Okay, you know, whatever the discounting is.” And so, you know, we expect that going to procurement, you go — a bit of a Kabuki show, and then, you know, you end up with some pricing. And that pricing is actually for the vertical has been set in the market somehow. So it’s pretty well understood for forecasting by the business, pretty understood by the customer, and it’s, kind of, motions you have to go through. Do you find in open source that that discussion is different because you are open source? Like, do you have more pricing pressure because it’s open source? Do you have more of a push-forward service component? Or can I think about it the same way I think about closed source?

      James: I think you can think about it the same way as closed source, if your model is, you’ve sold someone on a big new trend, you’ve helped train their organization to get there, and then you’re working on a fresh demand forecast of the transaction together. If you’re coming in and upselling support only, and they’re already installed, and all you’re upselling is a support model, that’s gonna be a lot more difficult. Because they already have it running, they already have it working at their scale, and all they want is, essentially, professional services by phone.

      Competition with larger players

      Sonal: What happens when a big company comes in and competes with you as a smaller, growing open source-based company?

      James: This is something that open source companies have to navigate, which is that sometimes larger companies will adopt the same software by name, if you’re not careful about how you position yourself.

      Sonal: Wait, what do you mean, like, they can literally just take your name even though you…

      James: It’s very popular for large companies, say, IBM or somebody else, to say, “We support X.”

      Sonal: Oh, I see what you mean. Okay, got it.

      James: Larry Ellison famously did it with Larry Linux. He tried to do it to Red Hat. They often don’t have that much capability behind it, but they can, at least, push a little go-to-market on it.

      Sonal: But wouldn’t your internal champion in the procurement office be able to tell the difference? Like, why would they even pick something else when they can get your product, which is what they want?

      James: This is why having a unique market position is important regardless if you’re open source or not, because I’ve seen cases where very large orders were suddenly stopped because another large company just quoted something that sounded similar for the same amount. Now, that can happen open or closed, but if you’re in an open-source dogpile, as I mentioned, word of caution, it’s especially rampant. That can actually drive your entire upside of ever being a half-billion-dollar a year software company — like, your probabilities go down really hard.

      Sonal: So how do you get out of it?

      James: The key is keeping a unique offer in the market, right. You don’t wanna be yet another distro of, you know, a certain famous technology that provides support. I think you wanna have a fully packaged differentiated approach.

      Martin: So here’s probably the most basic question in this space. If you have an open source project, it’s probably available online. So if you’re walking to somebody and you’re about to sign a $10-million ELA for license, what’s stopping them from just downloading and running it themselves?

      Sonal: Yes, I have that same question. I’m glad you asked that. I wanna know the answer to that too, James.

      James: You know, some of the origin of this discussion was, we were speaking on Twitter around this $100,000 or $200,000 contract value, and even some sub that, as sort of a valley of death. I think, if you go back to why the big software companies are, like, successful, they’re an extended part of those companies’ teams. So, I don’t really like the $100,000 a year relationship, because it’s very transactional. When you’re part of a big change like microservices in the enterprise, they’re gonna want advisory. They’re gonna want you there. They’re gonna want a team of two to three people that just live there. A key thing is when that $10-million transaction comes, they’re as much voting with, you know — they want you to be part of their extended team, part of their strategy, part of their relationship with you.

      Sonal: Yeah. 

      James: Because you have an expertise that’s been hard-won that they do not have.

      Martin: So I spent, you know, quite a while doing enterprise sales, and I’ve got this view, it caused this conversation that you and I had on Twitter, which I had claimed that, you know, in enterprise software sales, there seems to be this valley of death between, like, say, 30k and 150k. If you’re 20k or below, you know, you can call somebody up, and they can pay for it. And let’s say if you’re 200k and above, then you have hopes of supporting a direct salesforce and still have good margins. And then, in between that, you’ve got this valley of death, where you can’t really support a direct sale because you can’t pay the people. But, you know, if you’re trying to do something new and innovative, you don’t have account control and it becomes very transactional. So, do the dynamics of that change with open source sales?

      James: I think what’s happened is that open sources become how large enterprises wanna bet on big new trends. And then that opens the capability for the first time of open source companies — I’ll use the word — being high end. Meaning that IBM and Oracle are parking a bunch of good platform architects, as we call them, technical architects, at the account that explain the new things to that account. If you wanna get a $10-million relationship going, you’ve got to become that extended part of their team. That is why I jumped in on our original Twitter discussion around the $100,000 a year deal, and I was like, “Hey, actually, I think 100 is too low. Like, I would try to get to 400 to 1.5.” And the critical thing that happens at that [amount] is not that you extract more money. It’s actually that you can be a better advisor. Like, you can actually put talent on the ground, and you’ll find that these large enterprises, during times of big change, really value that talent on the ground.

      Martin: Interesting. So, this is a nuance, a new way of looking at the discussion. So, how much does your ACV have to be to support a salesforce and say, “It has to be at least 150, 200k?” You would say that it actually should be higher than that, because the goal isn’t just the margins on a direct salesforce. The goal is really strategic account control.

      James: Correct.

      Martin: And to get that, you need a deeper engagement. That’s very interesting.

      James: Yeah. And I think so many open source companies have died because they’d transact around the edges with, like, director level or technician level for support, and they never got to the CTO or the CIO and true strategic account control like the big players had.

      Why open source is expanding

      Martin: So, I’d love to get back to this question of, like, why now? Like, I’m just so curious. Which is, like, is it, like, have you figured out something that nobody else has, or is it that the enterprises now realize that they need to pay for this software, or is it something else going on? Like, why are we starting to see the Elasticsearches? Why are we starting to see the Mesospheres? Why are we starting to see these companies be successful?

      James: Well, if you look at the clock cycle of how often big new software companies get built, it’s not every day. Enterprise architectures only change so often, right? Like, if you were to try to sell a middleware offer in 2008, I don’t think you could have raised money. You would have had a really hard time, because none of the design patterns were changing. I think what’s happening now is that cloud is disrupting a lot of people’s approach to software infrastructure, to how they build applications. And so, this crop of open source companies is getting a chance to be the leader of a new thought, versus purely being a commoditization play.

      Martin: And so, you’re saying that it’s not inherent in open source, as I see inherent in the trend that’s going on, and open source is just becoming, basically, a requirement because of customer expectations in community.

      Sonal: But I have to push back on this one. Isn’t that inherent in open source, because that has to do with the nature of a community, the speed of development, the fact that you don’t have to necessarily go through a waterfall type of development process? I mean, isn’t there something here that’s inherent to open source, by definition?

      Martin: Fixing problems is fixing problems, whether you do it in open source or a closed source. And the reason that we’re seeing this rise in open source companies is not endemic to open source. It’s the fact that we’re seeing, actually, a transformation around developers and their aesthetic and the technologies.

      Sonal: Totally. I totally buy all of that.

      Martin: And then the question is, well, where does open source come into the play with it? It seems like there are intrinsic benefits to it, and there’s a level of expectation from the customer.

      James: Yeah. And I think that the commodifying open source companies did a really good job of normalizing open source as something that people did, and even creating an expectation that that’s the right way of going. And then, this wave of what you might call, like, the microservices generation, or the cloud generation open source companies, are actually catching enterprises with a new need, and a new, you might call it, hundreds of billions of dollar need. And suddenly, IBM and Oracle’s earnings are missing every time, and suddenly, these new companies are getting green shoots.

      Sonal: Okay, that’s fair. Just wanted to make sure that there wasn’t some inherent experimentation baked into an open source-based project that then leads to this more innovative type of…

      James: You’ll keep getting me to talk open source strategy, I’ll keep talking software strategy.

      Sonal: That’s great.

      Martin: I love it.

      Sonal: That’s great. That’s, like, what we need to hear.

      James: Sometimes people say, “Oh, well, why is enterprise software so expensive?” It’s expensive because, you know, the trust model of enterprise buyers is, you’re really gonna take care of them, advise them, ensure outcomes. You’re not just providing support. So what you’re really fighting for is the chance to be one of those new trusted architectural advisors, I believe.

      Sonal: Red Hat lived on the x86 shift.

      James: Red Hat kinda got lucky and found a commoditizing change, which was Unix to Linux. That’s a rare event to have a major, you know, chip system re-platforming sea change.

      Martin: So, let’s talk about Red Hat, because Red Hat seems to not follow — there’s a big transformation. Open source is a great way to do it because it gives you the edge with community, all the things, you know, all the bromides that we normally talk about. So, you take advantage of the transformation, you become a strategic advisor, etc. But if you look in the back, that’s kind of, like, the one success case doesn’t really follow that.

      James: For all of the time it’s been in market, Red Hat is still a $2-billion a year software company, right? If you compare it to what IBM, or Oracle, or SAP, or Microsoft, or any of the megas, they still haven’t caught that size of a trend, even though their prices, frankly, are similar to Microsoft’s now, like $2,500 an OS. So it’s not that they’re just down-market, it’s [that] they just didn’t catch a trend big enough to become a company that big.

      Sonal: Yeah.

      Martin: I see.

      James: Like, I don’t think they — you know, in the same way that the mainframe captured all of the world’s transactional processing, or Oracle captured all relational database apps and enterprises.

      Sonal: I mean, you’ve described this now a few times in this podcast, catching this trend, this wave, this need. That’s just product-market fit. Like, there’s a real market need for this.

      James: Yeah. My thesis is that, just saying “We’re an open source company” can distract you from software strategy.

      Sonal: Yeah, exactly. That’s great.

      Martin: So, what did OpenSolaris get wrong? I mean, like, you’ve seen this from many angles. Oh, there’s a pained look on his face. I’m not sure if I’ve kicked a wound.

      James: You know, it was a great learning experience for me because I think Jonathan Schwartz had a fairly early and simplistic open source strategy. And he was one of the first CEOs of a mega enterprise company to do this. So I watched this firsthand. And he was like, “We’re an open source company.” The problem was that Sun was still on the wrong side of all the trends. Like, it was actually late to x86. It was late to everything that was happening around Webscale. And so, open source actually only hurt us, in a sense, because, you know, working on monetizing OpenSolaris, one day, Jonathan said, “It’s all free. It’s no longer licensed. Go sell support contracts. Everyone will buy support.” But what happened was that, overnight, like hundreds of millions of dollars of revenue evaporated, and then it was a tactical challenge to go get people to sign up for it again. We didn’t have some strategic initiative to go talk to them about.

      Sonal: Which goes right to building a strong, good software company, to your original thesis.

      James: So we worked on that. And then the next thing I saw was, within the Spring group, there was something called tc Server, which is just cheaper middleware. And it only got to, say, tens of millions of dollars in sales and an average selling price of, say, 80,000, somewhere in that range, and it never caught a strategic new design point either. So when we had PCF, I really said, “Hey, this microservices refractory is actually a once-in-a-generation change. We could take a different tactic.”

      Martin: What I really like about your view on this conversation is, you’re basically saying, open source or not open source, you have to build a solution that solves a real problem, take advantage of a trend. It’s like a traditional software company and software sales, and then open source provides uplift, and etc. And often, open source has been viewed very differently, like, it’s got something magic that will save you. And so what you would see, is you’d see these incumbents that are desperate, and, like, in an act of desperation, the swan song is to release something open source, because they think that, somehow, that’s gonna magically save them.

      James: Community development is really important, but if you just develop a little community and it doesn’t change CIO, CTO — anyone that’s writing <inaudible> checks priorities, like, by definition, you didn’t build a software company that had an opportunity to be the big new strategic advisor and…

      Martin: That’s a critical piece

      James: …new architecture.

      Sonal: Right.

      Martin: Another thing that I like about your view on all this — it seems like it’s a very, kind of, planned, top-down view. Which, often, discussions around open source, to me, are very organic, which is, you know, you get a community, and that community will grow. And like, this is kind of very, kind of, like, organic progressive thing, and then you have some…

      Sonal: Well, this goes to my question about product vision, like, having someone to direct it top-down. But what I’m hearing is, the next Steve Jobs will be the Steve Jobs of an open source company, because he’s a great CEO, great product guy, great business. It doesn’t have to be open source or closed source. To your point, it’s building a great software company.

      Martin: Exactly right, which is like, listen, there’s an industry trend, you’re gonna build real value-add in that trend, and you’re gonna go from the top-down, make it a direct salesforce. And this is really about kind of top-down, like, strategy and planning, and not just, kind of, progressive community…

      James: A great thing that’s happened is because buying open source has become a normative behavior.

      Martin: Yeah, exactly.

      James: Which means that you don’t have to be shy about doing something audacious and asking for money.

      Martin: Push that even further, do you think open source has become a requirement, or is becoming a requirement?

      James: It’s close to a requirement, I think, for major new bets. Like, if you look at our biggest buyers, they don’t wanna get into what Oracle’s doing to them right now. Like, every year, they’re using less Oracle, and every year, they’re paying as much. That’s a very negative feedback loop.

      Sonal: Wait, explain why.

      James: ELA structuring, enterprise license agreements.

      Sonal: And what is that? Like, just kind of talk us through it.

      James: So what buyers are trying to escape from, and one reason they want open source as a protective measure, even if they don’t use it, is the right to keep using their software without paying more and more for it every year. Companies like Oracle have been out there charging more and more for the same usage over the last few years. So, they basically trap you into that and say, “Well, we can increase your unit cost even if you’re decreasing your unit consumption.”

      Advice for entrepreneurs

      Martin: Yep, that’s great. To get back this original question, which is, a lot of the constituents that listen to this podcast are entrepreneurs or aspiring entrepreneurs or existing entrepreneurs. So let’s say one of these entrepreneurs wants to create a company, Acme, Inc., and they have…

      Sonal: Acme, Inc., great naming there.

      Martin: Acme, Inc., that’s right, because I’m creative. So they’re creating Acme, Inc., they’ve got a software project they think is gonna change the world, and they have a decision to do open source or closed source. Is there a difference, or is it the exact same thing?

      James: I think we got incredible lift off of open source. For instance, IBM showed up and standardized, then, what we were doing, HP, SAP, other people. We got huge lift off of open source. Now, that doesn’t mean that you have to accept the usual low-end tarball and support contract business model. So I would probably not start a closed-source software company today, unless I had a very niche, novel idea that I felt no one else would try to do. Because quickly, there’ll be an open source alternative to what you’re trying to do if you do not open source it.

      Sonal: Okay. Last question then, you started off describing how you’ve been through, like, two, sort of, failures before you kind of struck the right model for building a great software company, not necessarily a great open source company. What advice would you give to entrepreneurs today who are trying to do the same thing or the next thing? 

      James: I think the tradeoff that you’re gonna make as an entrepreneur is to come out into an existing trend with a lot of people having already validated that. That was easy to get into that trend. It wasn’t a big risk. So, I think the challenge for entrepreneurs is trading off, like, a vision of change that will happen over the next couple of years that you’re gonna grow into, versus just joining an existing parade around a standard open-source community.

      Martin: Yeah. And I would add to that, things that you could tangentially apply to previous architectures don’t necessarily carry forward, right? I mean, like, often, we’re like, “Oh, this worked for VMs, therefore, it works for containers.” And so I do think that…

      Sonal: Oh, interesting.

      Martin: …you really need to be piped in the nervous system of the evolution of the market, and you really need to be bold about, kind of, going after new solutions that are part of the evolving landscape.

      Sonal: I mean, it’s a first principles way of thinking.

      Martin: Exactly.

      Sonal: It’s sort of like saying, “Don’t derive it from what happened before. Like, think about it from scratch.”

      James: Martin said one of the most brilliant things, which I was sitting at home, listening to the podcast, cheering on, which he said, there’s so much metadata and microservice now that you can change the way you think about networks. That is the fundamental shift, that if you catch something like that, then you have a game-changer.

      Sonal: That’s wonderful. Thank you for joining the “a16z Podcast,” James.

      Martin: That was great.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • James Watters

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      Build Your Personal Brand

      Alex Constantinople, Margit Wennmachers, and Hanne Winarsky

      Your brand, says head of a16z marketing and Outcast Agency co-founder Margit Wennmachers, is what people say about you when you’re not in the room. And it’s going to happen, whether you choose to have an active part in it or not. But what does this mean at an individual, not just company/product level?

      In this episode of the a16z Podcast, Wennmachers and Outcast CEO Alex Constantinople — both longtime veterans of public relations and building executive profiles — de-mystify what having and building a personal brand takes. It’s not only about “thought leadership”, either… a personal brand can also provide a filter for choosing what to do (and what not to do), as well as define your aspirations for where you want to go next. Even if you cringe at the idea of putting yourself in the spotlight.

      This conversation, moderated by a16z partner Hanne Tidnam, was recorded as part of the BreakLine Tech program for military veterans, an immersive education program for veterans transitioning into new careers (including a week of talks and courses hosted at Andreessen Horowitz, some of which can be caught here).

      Show Notes

      • Defining what a personal brand is and how to get started [0:30]
      • The importance of storytelling and tension [5:20] and why everyone needs a personal brand [7:19]
      • Examples of effective personal branding and the importance of authenticity [10:38]
      • Logistics around creating a brand [13:55] and how to know when it’s working [17:12]
      • Managing mistakes [19:29] and audience Q&A [23:45]

      Transcript

      Hanne: Hi, I’m Hanne, and welcome to the “a16z podcast.” The phrase “personal brand” is something of a cliché, but we all know we’re supposed to have one. So what does it really mean, and how do you go about actually creating one? In this episode, a16z’s Margit Wennmachers and Alex Constantinople, CEO of The OutCast Agency, both break it down into basics and also give us a sense of nuance on how best to think of a personal brand. This podcast was recorded as part of the BreakLine tech program for military veterans.

      Defining “personal brand”

      So, I just thought we would start with a really basic question, just to lay a little groundwork, which is, what do you think a personal brand actually is? How would you define it?

      Margit: I think, in a nutshell, it’s basically what people think or say about you when you’re not in the room. That’s how you should think about your brand. What is your reputation? What is the association that you occupy in someone’s mind? And so, that’s in a nutshell what it is. If you think of — companies are easier than people. If you think of Apple, you probably think of design and elegant products. If you think of Virgin, you probably think of irreverent and fun. Like, those are the brand attributes that you think of, not even consciously, necessarily, and that really what defines a brand.

      Alex: Yeah, I mean, the good news is, just being conscious about it actually will help you. So, I think, while it is a huge part of what others say about you, I do think it’s what you choose to put out into the world as much — and is actually more important.

      Hanne: So let’s say you’re starting totally from scratch. You know who you are, you know what you’ve done, you know what your resume says, but how do you go about — step one to finding what your personal brand is?

      Alex: What we usually do is we’ll have an executive come in and really just do a whiteboard session. And we really start with, “If I talk to your neighbor, if I talk to your parents, or your partners, or your best friends, or your co-workers, what would they say about you?” And we find that’s an easier entry point than if I say, “Give me adjectives,” it feels weird. Like, “I am the smartest, the prettiest, the most fabulous…” Like, it’s harder to get it out of people because it’s awkward, right, to be, like, “This is who I am.” But this is how we usually just really start.

      And then the next question, the most important one probably, as you’re thinking for yourselves, is also, “What kind of leader am I? What is it that I want to put out into the world and have people see? How do I want to be?” And this can be aspirational, all of this, by the way. It might not be who you are today. You might feel like, “You know, I’ve gotten feedback.” I mean, my 360s that I’ve gotten from GE, past to now, are hilariously the same. It’s sad. I’m not happy about it, but there are some feedback points in there that I’m like, “I can’t get rid of that,” you know?

      And so, I think being conscious about it, like — what are some of those things, the way I lead, that I want to be seen and then how do I get there — which is part B of this. And then really your expertise is a big part. How do you want to be seen out in the world? This is where more of you heard the phrase “thought leadership.” When you’re out more in the external world, what do you want to be known for? What are you really, really good at? What can you own as an expert? And then that can be subject matter. It can be super broad or very, very narrow and all of the above.

      And then the last for personal brand, I think, is really everything about you, because I find you can’t leave your personal stuff at home. You can’t leave that you might love the outdoors and you’re more the adventurer. It helps round up the picture. You love to read. You love to be with your family. Like, that is you. And if you come to a job, I find, without your full self, you can’t have the most value. And so we don’t leave that off as soft stuff. It’s really important that you are authentically you.

      Hanne: Another way of getting at this is thinking about story. When you’re saying all that, I’m thinking, like, “Well, that’s so much information.” I mean, how do you know what the story is that pulls it all together? What’s a good way of thinking about that?

      Margit: I think, to Alex’s point earlier, that is where you have a fair amount of control, right? Like, what are the anecdotes that you want to share, right? Like, what’s the part in your childhood that shaped you that made you join forces, or that made you the leader that you are. Like, you can control all of those anecdotes. If you think of, like, a very carefully crafted brand — whatever you think of the person or not, this woman Sheryl Sandberg — you all know who she is, right? Well, if you hear her speak, or if you read her book, or if you see her on TV, there’s always a story about when they were kids, she was managing her siblings, right? She put that out there, right?

      So, a good way to get at what your version could be is, if you read — take any of your favorite magazines and read a personal profile that someone has written about a business leader. It’s probably the most relevant example, or an athlete, or whatever — and, sort of, look at, “Okay, what would my version of that be? How would I fill out paragraphs 1, 3, 7, right?” And you see, once you dissect an article, it becomes not as black box voodoo-ish as it seems when you first think about brand, right? You go, like, “Oh, they have their family interests. They have their childhood experience. They have their expertise.” And you can deconstruct the story. And then if you take one of those articles, go like, “Okay, if I had to write a story about me, or if I wanted a story written about me, what would be in that story?” And that gives you a control over what it is, and it also helps you build the body of how you talk about yourself.

      Hanne: So are there things, though, that you think universally make a good story? You know, that you look for when you’re helping people do this, characteristics that you say, that’s…

      Margit: So, I think it depends. We haven’t even talked about — we’ve talked about what is your brand, how do you want to describe yourself. We haven’t talked about, like, how you put it out in the world, right? So that’s a whole…

      Hanne: Right, mm-hmm, which we want to come to next.

      Margit: Which we want to come to. But when you think of stories that other people tell about you, like a magazine article or something, they always want some tension. And that’s fine, as long as I think there’s a happy ending at the end, right? And tension can be anything from a tough childhood or a really tough mission, to the extent that you can talk about it, or, you know, what countries were you deployed in and whatnot, right? But, like, they all want some tension. They want the reader to go along and go like, “Okay, I want to read the next thing,” right? It can’t just be like, “Here’s my picture-perfect resume, and, like, yay.” Nobody wants to read that, right? Like, we don’t even want to read that.

      Alex: I was going to say like, if there are lessons learned — people really like what can you bring to somebody else. And so, I love people who put themselves out there and are a little more vulnerable. And I know that is hard. So being able to say, “I tried this,” or, “This was something I did that didn’t work, but I learned x from this,” I think is a great way to think about that particular tension.

      Margit: Just last night I was reading your profiles. I was like, “One, we shouldn’t be talking. You should be talking.” <Exactly.> But there are great stories in there, right? Like, one of you tried to land one of those planes, and it didn’t quite work out.

      Alex: Raise your hand if you tried to land a plane that didn’t work.

      Margit: That’s amazing.

      Man 1: <inaudible>

      Margit: You’re here, right? But I was reading that and, like — look, it stuck with me. So there are amazing stories. I thought these profiles were really, really interesting to read, and there’s a lot of good stuff in there, where I thought, like, “Oh, my God, they have a lot to work with.” And also, thank you for all the stuff that you’ve done.

      Why everyone needs branding

      Hanne: So, maybe this is totally obvious, but does everybody need a personal brand then?

      Margit: Yes. Soapbox moment.

      Hanne: Even if you’re interested in a job where, say, you don’t want to put your opinions out there that much.

      Alex: I do this with college graduates. Anybody who needs to tell — to be — they’re going out into the world doing something, I feel, like, “Shit.”

      Margit: If you have interactions with people, you need to think about this. And if you think of the startup world, here in Silicon Valley, most of them — they languish in obscurity. You do want to stand for something. You want to be remembered for something, you know, as much as there are a gazillion jobs out here, right? Like, everybody needs to go, like, “Okay, I want this person, because they struck me as such, such, and such and such.” And a brand doesn’t have — it doesn’t mean fame. I think people confuse brand with fame. If you have a powerful brand with the right 20 people, that may be it.

      Alex: Yeah, that’s exactly right.

      Margit: It doesn’t have to be fame. It’s not like — you don’t have to be on CNN or whatever. Like, no one’s saying that, but you just want to have a deliberate way of thinking about, “Okay, how do I want people to think of me?”

      Alex: I wish there was another name for it. I think, especially here in the Valley, personal brand, I found it was — I spent the majority of my time in New York and D.C. before I came out here, and personal brand, it was no big deal. Here, it’s sort of like, “Oh, I don’t want one of those. Like, that sounds too much for me.” The personal brand can turn people off because they — certainly if they came up through the technology and the engineering side, it’s very uncomfortable. It’s like, “I’ll go out there. I’ll only do my Fortune story or whatever…”

      Margit: They feel like they’re lying.

      Alex: “…if it’s good for the company, but I’m really not interested.” And they’re kind of missing the point, because, to Margit’s point, it’s not necessarily the next step — which [it] can be for people — then what’s the communications plan against your personal brand. And that can be, “Okay, start talking at these places and giving these kinds of speeches, and let’s work toward, you know, this kind of profile and this kind of publication that will help your business grow.” But I think for purposes of just any executive — and I’ve literally done them for professors, for, you know, scientists, people who you wouldn’t think this would be — but it really can work with anybody because it’s just a way to frame your activities.

      What we also say to a lot of executives, no matter what level — it really also helps you as a filter for your time. If the activities you’re doing — and you’re going to get asked to do a lot of things, right. If it doesn’t necessarily fit within what you’ve laid out for yourself, then I think it’s easier to say no. It’s like, “Right now, I’m focused on this.” You know, obviously, everyone has a favor for a friend that they’ll do, but I think it will help you focus and save your time. So that’s another really practical reason that we try to put this down. Now, it doesn’t mean it won’t evolve over time. You can look at it again in five years’ time, in two years’ time, in a year, and say, “Okay, now that I’ve been doing this…”

      Margit: Achieved that.

      Alex: Yeah. Is this good? Does this feel right? Because I think we do want to push ourselves all to be aspirational.

      Margit: The other side of the coin is, like, brand happens to you, whether you want to or not. Like, people will describe you in their heads. So, would you rather have some say in what that is, or do you just want it to kind of let it happen, right? So it just happens. I mean, think of business executives that you admire, or hate, or whatever. Like, you have opinions about them, right? And so would you rather shape how people perceive you, and have it be true to yourself and what you want it to be, or just have it happen, right? So just take control, like you always do with everything.

      Hanne: Are there some examples of people — I like that you distinguished between fame and brand — that you think, maybe, are not on the famous side, but did such a good job telling their story and establishing a brand?

      Alex: You mentioned Virgin already. I think Richard Branson.

      Together: Yeah.

      Alex: I do think he’s done quite a good job. The companies he’s built absolutely are from him. You don’t feel a disconnect. He doesn’t say things that then don’t show up in his companies.

      Margit: By the way, it’s the only business, I think, that has a brand that’s consistent across very different businesses.

      Alex: Yeah, yeah. He’s a horizontal growth, so it’s like, you know, mobile to hotels to airlines to…

      Margit: Trends. Everything.

      Alex: Yeah, it’s crazy, but it’s — that thread through works, and then the way he jumps out of planes and, you know, doesn’t have insurance or whatever the hell, like, seems to work for — you know, it absolutely works. It absolutely works.

      Margit: Another person I think who’s done consistently a good job of their brand is Warren Buffett. He’s just very authentic. He’s got that folksy style, but he’s also smart. You know, he shows up consistently, and I think he’s done a really lovely job of managing his brand. And I’m not even sure he’s consciously doing it. Maybe he’s just lucky and gifted. Sometimes people are more gifted than others. Another person who actually doesn’t even tell her story, but I think has done a good job, is Angela Merkel over in Germany. I think she’s just like no-nonsense, right? It’s not a flashy brand. It’s not a “let me use my feminine charm” brand at all. It’s just, like, you know, walks the line.

      Hanne: She is who she is.

      Margit: She is who she is, and she’s just like boom. She keeps marching, right? So I think she’s done a good job. And then the example of someone whose brand has changed a lot for the better will be Bill Gates. If you — you know, some of you are too young but, like, he was just, particularly in Silicon Valley but I think widely, hated because of their hardcore business behavior. And now he is one of the most admired — and rightly so, one of the most admired human beings. Now, it’s easy when you have that much money to throw at the problem, but still, a lot of people have quite a bit of money and don’t bother to try and improve the world. So I think he’s done a really good…

      Alex: That’s a good one.

      Hanne: That makes me think of — you mentioned authenticity and, like, the role that authenticity plays. I mean, how do you avoid feeling overproduced or over…

      Margit: I think it starts with, if you’re trying to portray something that you truly are not. So let’s just say you are hardcore competitive. Then don’t try and make your brand be, like, “I’m a little puppy dog,” right? It’s just, like, not going to work. Just own who you are, right? And I’m sure there’s an okay version of who you are, and own that, right? So that’s step one, authenticity.

      And then Facebook, and Twitter, and Snapchat, and whatever else, they just demand authenticity, because it’s so easily detectable if it’s someone else doing the writing, or if these photos are too curated. From a content point of view, make it who you are. Like, everybody has features and bugs, in Silicon Valley parlance — like, “I have a lot of bugs, but you’ve got to find the place where the features are valued,” right, and you’re going to be successful in those jobs and not in others, right?

      Branding logistics

      Hanne: Okay, so let’s talk about logistics a little bit, and platforms. You’ve made your list of adjectives. You know, you’ve figured out you, sort of — how do you actually go about getting it out there? And, like, are all platforms — do you have to be on all platforms all the time and…

      Margit: Well, I mean, that’s an entire book of a conversation, but to start with, let’s just assume you’ve done not just the adjectives but also, like, “Here’s my story,” right? Here’s the biography that is not, sort of, your official resume that you send out in the world. I would start with — if it’s something you love — if you’ve done a lot of speaking as part of your work, and if it’s something you love, like, go to town. Try to get the TED Talk. But don’t try to get the TED Talk if it’s not something you really love, because the worst thing that you can do is just, sort of, do a very high-profile thing and then just fail at it miserably. A, it doesn’t feel good. And then, B, it just don’t you any favors. So I would start, if it’s something that’s totally natural to you, I would start with something really small and comfortable. I don’t know, it could be your alumni newsletter. It could be very, very small.

      And then, the other thing I would say is, not every medium is for everyone. So I’ll use Marc Andreessen as an example. If I do Q&A, he’s brilliant. He’s just very good at the repartee, the question and answers, being quick on your feet, getting to the heart of the matter. He talks fast, and the whole thing works, right? Just find what you are. If you are good at speaking, speak. If you’re good at writing, write. Now, if you’re good at speaking, you still need to write, because you want to make sure that what you say is, like, really deliberate and whatnot.

      But, like, everybody is different. There are things like LinkedIn and Medium, right, where you can share things like what Alex was saying, like, lessons learned or tips or, you know, like those kinds of things. Then obviously there’s press, which is the least controllable, because whatever you say, it goes through their filter and, like, they end up what gets used, and how, and all of that. So I’m sure you know the pitfalls. But it also is, in some ways, the most credible, because it’s not just you doing your own talking but a third-party, and they have their readership, and whatnot. But there are all kinds of options.

      Alex: I will bring it back to — in case none of that is where you are, right, it’s actually — within a job, it’s what activities are you doing and what are those — are you doing the kind of work that you want to be doing? Are there projects that you want to be on? I think there’s also ways to use this for your advantage within a company, or within your environment, and maybe it is more community advocate as well on the side, and then what are you doing to do that, right? You know, do you want to sign up for something? Do you want to participate in a non-profit? Like, whatever those other things are, that also can be included. So I think there’s a quieter way, also, to think about the execution of a personal brand exercise that can be — how do you show up wherever you are?

      Margit: Just to add another thing. It can also be, like, maybe you want to create your own personal network. Let’s just say you’re here, you have a job. It could be just, like, you corral a bunch of people and you have dinners. It shapes — as Alex is saying so eloquently, it shapes your activities and also what you say, and what you focus on, and what you want to impart.

      Hanne: How do you know when it’s working? I mean, is it followers? Is it, like, getting places published? When do you know, like, “I’m telling the right story”?

      Margit: I think it’s —companies spend millions of dollars doing brand studies, and they’ll do things like sentiment analysis, and Twitter followers, and all that kind of stuff. I think you know when it’s working, and I think you would know when it’s not. And it sounds like a pat answer, and maybe Alex can help me refine it, but are you working in the right job? Is that fulfilling? Do you feel like you’re connecting well with people? Are you spending your time on activities that you enjoy? Do you feel like your expertise is valued? To me, it’s like, are you working on something that you think is important, that’s larger than just yourself? And do you feel like you play a meaningful role in it? If you keep running into trouble, or if you keep not interacting well with people, then, yes, then I think it’s time to revisit it and go like, “Okay, what’s not working here?”

      Alex: I actually wrote down three things off of mine, that I wanted to use as a temperature check, which I keep looking at. So, you could have your own version of this but mine was, “Am I growing and developing?” So, actually, one of the reasons I took this job is that at first I was like, “No thanks,” and it was just, like, pre-“Lean In” territory. And I was like, “I can’t.” I just had my third kid. It was a surprise. Like, oh, my God, and then running a company? I don’t think so. I’ve never been trained for that. And then I was like, “No, this is — this matches growth and development. I’m going to push myself. I’m going to throw up probably every day, but that’s okay.” I cried a lot and, like, did “St. Elmo’s Fire” with the curtains a lot. Totally true for the first year, but I’m over it now.

      Growth and development was one. Adding value was a big one because that is — and I have two versions of that, which is — am I able to do what I do best at the job I’m in? Am I bringing everything? Are they accepting what I’m giving, basically? And that was also the personal part, and I actually think I’m successful because it’s all of me in that whole brand platform page.

      And the last one is just the fun, is my thing. You may have another one, but for me, I think, at almost 48, [I’ve] just been like, you know what? I am not working with people even on your crappiest day — I was going to say shittiest, but I’m trying to work on — on your crappiest day, that you can’t have a little bit of a laugh, or be like, “What the F is happening?” you know or whatever. And you just have to have that. So that’s my thing. So, you will have your own things, but I think that’s another way of thinking about it in the frame that you asked.

      How to handle mistakes

      Hanne: So what if you mess up, on a less happy note? What if you put something out there, and then you’re like, “Whoops, that totally doesn’t feel like me,” or you get a bad reaction? What then?

      Alex: You have a famous phrase, “Never waste a crisis.”

      Margit: Yes, never waste a crisis. It’s my way of coping. So, there’s a company version of this. If you mess up, like, how do you handle yourself, right? What are you doing? Because there are no secrets. We all know this, right? In theory, we all know this, and then we try to forget it when it applies to us, but there are no secrets — particularly not if you’ve tweeted something. It’s just, like, own it. Own it and move on. Just own it.

      Alex: Interesting enough, with the PwC thing from the Oscars, right? And all the coverage was, they are taking it on the chin big time. The chairman actually came out and quoted about it. I always appreciate, and I’m sure you do as a regular consumer — think of brands that are messed up, whether it’s a food brand and something happened, or, you know, just saying, “We did this. We’re sorry. Here’s what we’re going to do to fix it.”

      Margit: And then actually do it.

      Alex: Yeah, I mean, you probably already tell your kids that. Like, just own it and say you made the mistake. I mean, you’re only going to get as much trouble if you make eight lies and make me hunt you down.

      Margit: The thing about the human condition, we want to forgive. We just want to feel heard. We want to feel heard, and then we’re ready to forgive. But if you’re lying to us…

      Alex: You cannot get that.

      Margit: …then we get very needley and obsessed and whatnot.

      Hanne: In your own experience in building your own, you know, personal brand, what do you felt like was the hardest, or what was the most challenging for you?

      Alex: I would say the hardest and most rewarding was coming out here and not knowing anybody. I mean, my whole network was a completely different network, and I moved here, much like I think you guys are. And that was just hard, because it was, sort of, this blank slate of like, “Nobody knows me.” So it’s kind of awesome.

      And what do I want to be? So, starting over and making that transition, I think, can be very challenging but incredibly rewarding and you just have to be patient. The best times, coming out of that, I was just extremely thoughtful, and I’ve never made a “mistake” in my career yet so far. I did a lot of due diligence. I really thought about, you know, what kind of company do I want to work for, what brand, you know, is it. Like, how will my story — I’m not a planner so I’m not, “My 5-year plan, and my 10-year plan, and I’m going to be this, and I’m definitely not going to run for president in 2034, and holy hell.” But, I do think I’ve been along the way — to, sort of, combat that scaredness about it — just trying to be really thoughtful, and not rushing a decision, or not rushing into it, and not looking at a whole company, or not looking at the people that I’m going to be working with, and the kind of work, and can I be successful.

      Margit: So mine was — when I was running OutCast, we sort of had made a decision — it is going to be all about the clients, and we are not going to be out there and vocal. Maybe that was my excuse for not doing anything. But, like, my belief was, you don’t ever want to be in the news and have your client go and like, “What is she spending her time on while I’m paying?” Like, that just didn’t sit right, so I kept a very low profile. I basically did nothing. And then when I joined here, Marc, essentially, sort of, challenged me. He didn’t force me. He said, like, “I would highly encourage…” He’s very convincing. “I would highly encourage you to, like, up your brand profile a little bit.” And that was really weird. Like, it was so ironic, right? I’m sitting here. He’s going like, “You should work on your brand,” and here I am hiding in a corner, right? So he called me on it.

      And it was really difficult at first. So, I did things that were comfortable. I did dinners. I did dinners with reporters. And, like, somebody wrote a story on me out of that, which we didn’t work on that. It just kind of happened. It sort of happened organically. And then I always have, like, my happy home place. Germany — the Germans, like, want to talk to me all the time, because there’s so few Germans in Silicon Valley, and there’s all this tech tourism happening now. So if I want, like, an easy win, I’ll just go talk to the Germans and it’s like, “All right, fine.” But, like, that’s what I was saying. Like, find where you’re comfortable, right, and work your way into it. And it doesn’t have to be pressed, as Alex was saying. Find your way where you’re comfortable and, kind of, worm your way into it.

      Hanne: That’s a great note to end on, and we’ll take some questions. If anybody has, ask away.

      Audience Q&A

      Woman 1: Thank you very much for being with us this morning. Most of us were transitioning out of the military, right? And so we’re in the space of, somewhat recreating ourselves, trying to, you know, downplay — even though we’re proud of our achievements in the military, you’re trying to connect the dots where people see you being in an executive space, or being in the tech industry. So while we’re transitioning, is all the advice you gave the same? And then — or, also, maybe when you get more specific, of where specifically [there’s] a company or industry that you want to go into, how do you shift? How is that brand shifting happening? And can you do this by yourself, or is it something that you’d actually need to hire someone?

      Alex: You can definitely do it by yourself. I think the interesting thing is, on the stories, it’s what translates. It’s — what [are] the activities in the work that you did. In your military experience, a lot of the leadership skills in general, without being very specific to what each company does and what you’ll need to do in that company — finding those bridges of the work that you did, and the kinds of teams that you ran and oversaw, project management. Like, take those very basic things that are core to any leader anywhere, and map those for people just with your experience.

      Margit: Yeah, and I would say — I mean, you’re going to laugh, and rightfully so. But Silicon Valley thinks of itself as a place of disruption, which means there are uncertain environments that are wobbly. They can shift any second. And a company that’s hot now is not tomorrow and whatnot. And it’s full of people with engineering degrees, but not a lot of actual, sort of, real-world experience. So what do you guys do? You guys go into uncertain environments and make stuff work, basically out of nothing.

      So, I think that’s highly, highly applicable, and so you just need to find out the specifics of how you’ve led, and explain those in plain English. But, like, we need so much of that, because a lot of the folks here, they are running large companies but, like, they’ve never run a thing. They’re out of a dorm into their new dorm, with kombucha and massage tables. It’s a little mind boggling. So, someone like you coming in there is like, “All right, people, here is how we’re going to go…” is a thing of beauty, and I think that should be highly, highly transferable, and desired.

      Man 2: Thanks for both of your time. The idea of who you are is much more multifaceted than just, “This is my brand, this is what I want to be seen for.” Like, it can be situationally dependent. It can change on your life circumstances, and it can — you know, I may need to be a jerk in this situation, and that’s who I have to be, [and] in this situation I’m not. How do you encompass all of that authentically into one brand without having to, like, hide this side of yourself?

      Margit: Well, look, the brand is not trying to prescribe every detailed behavior in every situation, but I think — you know, having to be a jerk in a situation, that’s just sort of adjusting your management style, right? But, I think if you have three or four —three, we like three — brand acquisitions, it gives you a well-rounded body of, like, the essence of who you are. It doesn’t describe every behavior. And it’s also not static. I mean, if you looked at me funny when I was a teenager, I would be blushing, and I rarely spoke. And I can speak now, even in a different language. Go look at that. So, it’s not static as I think we might have made it sound.

      Man 3: I’m finding that it’s okay to be associated with a startup that fails. It’s actually positive for a lot of people, but it’s very negative, it seems, to be associated with a stolid, old-fashioned company who may be successful, but, if you go there in your career, you’re quickly known as one of “those guys.” Not good enough to make it at a, you know, high growth — is that a real concern, or is that something that we should ignore?

      Alex: I so lived this. This is literally where — I came here. I got to — yeah, my first job since moving here was with “WIRED” magazine. So, that was, sort of, my first, kind of, couple years, which was great, and I got to learn the space. And then I got to OutCast, and it was like my GE-ness — because we worked on startups and [they] were like, “Ugh, ooh, how embarrassing for you, basically.” And I think that was part of my year of feeling horrible, like, all this stuff I learned. And then I realized, “You know what? All of the stuff I learned through osmosis, through being in boardrooms, or just my experience traveling around the world is actually bigger.”

      So, I had to, sort of, move from feeling really bad about myself about it, right, and that it was an albatross. And I have to say, I over-rotated a little bit in the beginning. I tried to bring, like, too much project management, or too much process, to the company, I think, in the beginning, and then I found my way. You know, by people saying, like, “This seems too much,” right? So, I learned also a lot of, like, not necessarily my way was always the right way. So learning to be flexible like a startup, I think, was hugely valuable. But you will get that. A lot of startups don’t have that experience of how to run, you know, a big company, and that is actually what they all aspire to so it’s sort of ironic.

      Margit: I think there’s the chatter, and then there’s, like — we have an executive talent team. When a startup gets a certain level of momentum, they actually do want someone who has sold to big customers before, or who has worked in a big security department before. Like, they do do that. There’s, like, the what’s cool and there’s like — Forbes does a list of, like, the 30 under 30, and the 20 under 20. And, like, nobody does the 60 under 60, right? But, like, you do — you know, I think in the real world, once companies get to a certain scale, they actually do want the experience, and they do want, sort of, the big company-ness.

      Alex: But I do think you’ll pick up, when you’re interviewing or talking to these companies — you easily can pick that up, I think. There are some founders who aren’t very good at the “this is the way I did it at Microsoft,” and you could feel it very quickly, they’re not interested. And then it’s just, you know, “Fine, good to know,” or not your place or maybe that is your place because you don’t want to be like Microsoft.

      Hanne: One more maybe?

      Woman 2: Hi, thank you so much for your candor, by the way. It’s very refreshing.

      Margit: We only have one version. It’s the brand working.

      Woman 2: So, we’ve had a lot of feedback on translating military skills into civilian skill sets and things like that. And really what that boils down to is branding, in a way. And one of the things that I think is pretty universal throughout the military is the ability to be, you know, an athlete, and do a ton of things all at the same time. I think the problem with that, with our personal branding efforts, is how do you portray the fact that, “Hey, I have a lot of different skill sets,” without coming across as contradictory? I think my concern is just that, if we do brand ourselves as this athlete that can come in and do a lot of things, it’ll come across as we’re, sort of, a jack of all trades and a master of none. You did kind of touch on it, but how do we keep from being, I guess, pigeonholed into, like, the standard military, “Oh, you’re a military member, so you need to do this specific thing and kind of…” Does that make sense?

      Alex: Would you say that you were wide and deep though?

      Woman 2: Yeah.

      Alex: That’s how I would phrase it, right, that you can go wide. Like, wherever you’re going to go, you’re going to be successful, because you know how to go deep, and then think of maybe, you know, two examples where you did that. Like, we say that all the time, even with what we do. Like, our portfolio companies we work with is — everything from Patagonia to Amazon to Airbnb. <inaudible> company, right? So, when they came to us, they were like, “Well, do you have life sciences?” and we were like, “No, but…”

      Margit: You know that.

      Alex: “…we know what to do.” We’ll learn really quickly on life sciences. Like, we get up to speed. We know so many industries, then we go deep. You know, we know there’s a way to get smart, and then we can go deep but we are not — and then we own it, by the way. We say, “We’re not a life sciences agency. If that’s what you want, we can make a recommendation for you, but I can’t pretend to be something I’m not.” And then usually they’re like, “Ooh,” or they’re like, “Thanks, we’ll be moving on,” and I’m like, “Okay, bye.” So that’s okay. Like, I would rather say that than be like, “Yes, we can do that for you.” And then you get in there and you’re like, “Shit, really there’s no way I can do that,” or public policy…

      Margit: Only so much winging it.

      Alex: All right, thank you.

      • Alex Constantinople

      • Margit Wennmachers is the head of marketing and content at a16z, where she also advises entrepreneurs on their communications and marketing strategies. Previously, Margit cofounded the The OutCast Agency.

      • Hanne Winarsky

      Brains, Bodies, Minds … and Techno-Religions

      Yuval Harari, Kyle Russell, and Sonal Chokshi

      Evolution and technology have allowed our human species to manipulate the physical environment around us — reshaping fields into cities, redirecting rivers to irrigate farms, domesticating wild animals into captive food sources, conquering disease. But now, we’re turning that “innovative gaze” inwards: which means the main products of the 21st century will be bodies, brains, and minds. Or so argues Yuval Harari, author of the bestselling book Sapiens: A Brief History of Mankind and of the book Homo Deus: A Brief History of Tomorrow, in this episode of the a16z Podcast.

      What happens when our body parts no longer have to be physically co-located? When Big Brother — whether government or corporation — not only knows everything about us, but can make better decisions for us than we could for ourselves? That’s ridiculous, you say. Sure… until you stop to think about how such decisions already, actually happen. Or realize that an AI-based doctor and teacher will have way more information than their human counterparts because of what can be captured, through biometric sensors, from inside (not just observed outside) us.

      So what happens then when illusions collide with reality? As it is, religion itself is “a virtual reality game that provides people with meaning by imposing imaginary rules on an objective reality”. Is Data-ism the new religion? From education, automation, war, energy, and jobs to universal basic income, inequality, human longevity, and climate change, Harari (with a16z’s Sonal Chokshi and Kyle Russell) reflect on what’s possible, probable, pressing — and is mere decades, not centuries, away — when man becomes god… or merges with machines.

      Show Notes

      • How humanity is focused on changing itself rather than the external world [0:42]
      • The illusion of the self and the problem of tribalism [6:32]
      • How collecting personal data could lead to hyper-personalization [12:16], and even “religions” based on technology [20:53]
      • The future of work and UBI [25:44], and what humans will be good at in the future [32:17]
      • Political questions [36:27] and a look to the future [39:51]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I am Sonal, and we’re very honored today to have as our special guest Yuval Harari, who teaches at the Department of History in the University of Jerusalem and specializes in macrohistory and the relationship between history and biology. He’s the author of “Sapiens,” which is a mindbogglingly good book, and now has a new book just out, “Homo Deus.” Did I pronounce that properly?

      Yuval: I use the Latin pronunciation, which is homo de-oos.

      Sonal: De-oos. Okay.

      Yuval: But you can say homo dee-uhs.

      Kyle: I say the really bad, like, non-accent dey-uhs.

      Yuval: Dey-uhs is great. Yeah.

      Inward human evolution

      Sonal: That, by the way, was Kyle’s voice, who is also joining us on this podcast. He’s on the deal and investing team and covers a lot of the technology like drones, AI, and a bunch of other stuff. So just to get things started, we talk a lot about innovation and technology, and I’ve always wondered what’s the simplest definition of technology and innovation. And reading your book, “Sapiens” in particular and then “Homo Deus,” the thing that really struck me is that technology is the greatest accelerator humankind — in fact, the evolution of all the species on earth — has ever seen, because it allowed us to essentially bypass evolutionary adaptations where we could become seafarers without having to grow gills like a fish, for example. And so that is an incredibly powerful idea, but that’s non-directional. Given that your new book and your work, essentially, the first phase was talking about organic history of our species, and your new book is shifting to a more inorganic version, I’d like to hear what drove that shift.

      Yuval: Well, I think that so far, for thousands of years, humans have been focusing on changing the world outside us, and now we are shifting our focus to changing the world inside us. We have learned how to control forests, and rivers, and other animals, and whatever, but we had very little control over what’s happening inside us, over the body, over the brain, over the mind. We could stop the course of a river, but we could not stop the body from getting old. If a mosquito annoyed us, we could kill the mosquito. But if a thought annoys us, we don’t know what to do about it. Now, we are turning our innovative gaze inwards. I think the main product of the 21st century will be bodies, and brains, and minds. We are learning how to produce them. And as part of that, we may also for the time — not only in history. For the first time in evolution, the evolution of life, we may learn how to produce non-organic life forms.

      Sonal: That’s amazing.

      Yuval: So after four billion years of evolution of organic life forms, we are really on the verge of creating the first inorganic life forms. And if this happens, it’s the greatest revolution in the history of life since the very beginning of life.

      Sonal: What do you mean by inorganic life forms? Because in your book, you draw a distinction between biological cyborg and nonorganic. Are we just gonna be, like, living in a network? Is that our identity, then? Is that who we are? Like, what do you see?

      Yuval: It could be something that exists only in cyberspace. I mean, you hear a lot of talk about uploading consciousness into computers, or creating consciousness in computers. It could be life forms in the outside world, but which are not based on organic compounds. It can go any of these ways, but the essential thing is, it’s no longer limited by organic biochemistry.

      Sonal: Evolutionary psychologists, biologists talk a lot about our hands and the formation of our hands as tools. One thing that’s happened to me, anecdotally, is as I use my mobile phone more and more, my hand muscles have literally atrophied to some extent. I know this because I started taking notes again instead of on my phone to be polite in meetings, and my handwriting is literally — I used to win awards for handwriting, and now it’s like chicken scratch.

      Yuval: But it’s much more extreme, because for four billion years, all parts of an organism had to be literally in the same place for the organism to function.

      Sonal: Oh, right. Like, physically — like, in a single entity.

      Yuval: Physically connected. I mean, if you have an elephant, the legs of the elephant must be connected to the body of the elephant. If you detach the legs from the elephant, it dies or it can’t walk. Now, as inorganic life, there is absolutely no reason why all parts of the life form must be at the same place at the same time.

      Sonal: That’s mind-blowing.

      Yuval: It can be dispersed over space. This is something that for four billion years was unthinkable, and it’s just around the corner.

      Sonal: We’re essentially already uploading ourselves into the cloud, online social networks, in the World Wide Web. That’s actually replacing writing as a major artifact. That’s our new collective history. One of the consequences of that is it changes the dynamics of what becomes real and not real, and it reminds me of this famous story from Ray Bradbury called “The Veldt,” which basically is this story where there’s a virtual world that these two kids sort of enter, and they end up killing. And you ask a similar question in the book. You give the anecdote of Jorge Borges’ short story “A Problem,” and the story of Don Quixote. It sort of is this blending of delusion and reality.

      Yuval: The question is what happens when our illusions collide with reality. And with humans and human history, you see more and more that our fictions and illusions are more powerful, becoming more and more powerful.

      Sonal: <inaudible> say fake news. This is a big debate that’s playing out right now in the United States.

      Yuval: Well, you know, it’s fake news when we — with all this idea of the age of post-truth, I would like to know, when was the age of truth?

      Sonal: That’s my question. I totally agree with you.

      Yuval: Was it the 1980s? Was it the 1930s, the 19th century?

      Sonal: It never existed, right?

      Yuval: I mean, as far back in history as you go, what kept humans together in society is belief in shared illusions and shared fictions.

      Sonal: Imagined realities or imagined orders.

      Yuval: Yes, imagined realities, like when you swear the U.S. President to office, he swears on the copy of the Bible. And even when people testify in court, “I swear to tell the truth, the whole truth, and nothing but the truth,” they swear on the Bible, which is so full of fictions, and myth, and error. It’s like you can swear on Harry Potter just the same.

      Sonal: Some people do.

      Yuval: Some people do, that’s true. When, for thousands of years, human societies have been built on shared fictions and shared illusions, and there is nothing new about that, it’s just that with technology, actually, our fictions and illusions become more powerful than ever before.

      Sonal: Invisible to, I think, one another.

      The illusion of the self and tribalism

      Kyle: One of the illusions that you talk about being broken down by the advancements in science and technology is the illusion that we’re all individuals. Free markets and capitalism is the idea that there’s, like, a bunch of products that appeal to you as an individual, and they try to put those individuals into buckets and market towards them. And, actually, it turns out that scientific breakthroughs show that, actually, there isn’t just this, kind of, one individual you that accumulates through all of your experiences. Your brain is just kind of spitting out a lot of things. Maybe it’s deterministic, maybe it’s random, maybe it’s probabilistic, but you don’t necessarily have control over that. And so if you don’t have control over the desires — that your brain is spitting out the random thoughts, how much of any of that is actually you? And so, what are the implications of that?

      Yuval: I think what we are seeing is the potential breakup of the self, of the individual. The very word individual means, literally, something that cannot be divided.

      Sonal: Indivisible.

      Yuval: Indivisible. And it goes back to the idea that, yes, I have all kinds of external influences, and my neighbors, and my parents, and so forth. But deep down, there is a single indivisible self which is my authentic identity. And the way to make decisions in life is, just forget about all these external disturbances and try to listen to yourself. Try to connect to yourself. And the idea is, you just need to do whatever this inner voice tells you to do. But science now tells us that when you look inside, you don’t find any single authentic self. You find a cacophony of different conflicting voices, none of which is your true self. There is just no such thing. And even in the 20th century, the big fear for individualism was that the individual will be crushed from outside. Now, the threat comes from the opposite direction. The individual will break up from inside, and then the entire structure of individualism, and democracy, and the free market — it all collapses with the individual. It all collapses with the self.

      Sonal: Or, just one alternative possibility, because this is actually what struck me most when reading “Sapiens,” and then reading “Homo Deus” afterward — is that the big theme of “Sapiens” was this great unification of humankind, and being able to collect people into empires, nation-states, outside of these, sort of, hunter-gatherer tribes. And now when I look at what’s happening because of this mass coordination online, you’re now seeing this return to tribalism in some ways, I would argue.

      Kyle: Well, that’s, like, what the value of shared illusions are, whether it’s religion, or the idea that we’ve got this free market system but some safety net to keep it all functioning and keeping anyone from being exploited. The point of having that shared ideology or that shared illusion is, you get to pretend that we all care about the same thing, that we’re all coordinated towards the same goals.

      Sonal: Right. Now, though, because of the internet, you can actually identify what the same thing is at a very micro-targeted niche level in a way that was unprecedented. No longer where you were born, to your point, physically located. It could be now — your political beliefs. It could be your belief about, you know, if you’re a fan of Harry Potter. Are you a Slytherin or a Gryffindor? Like, it could be any of those things, and people collect into new tribes. And I find this fascinating because you do see sort of this return to the past, not in a pastoral way, but you’re seeing this coming full circle.

      Like, you know, the Industrial Revolution created adolescence. Are we gonna go back to a world where you don’t need adolescence again? You needed banking credit. Are we gonna go back to a world where, because of online algorithms and new information sources, you don’t need that version of a credit score. You can go back to this trusted personal manager who essentially knows what he needs to know in order to invest in you as a risk. So I always wonder in this context if this is another thing to think about, not just at an individual level, but sort of a return to tribalism, especially lately.

      Yuval: The present stage of a new nationalism or tribalism — I think it’s just a phase. It’s a backlash against globalization. And the main problem — it doesn’t have any solutions to the deep problems of the 21st century. All the major problems of humankind in the 21st century are global in nature. It’s climate change and global warming, it’s global inequality, and, above all, it’s technological disruption. The implication of the rise of AI and bioengineering and so forth — you cannot solve any of these problems on the national level. The nation is simply the wrong framework for that. And, therefore, I don’t think that nationalism really has relevant answers to the problems we now face.

      Sonal: I agree with you.

      Yuval: So I don’t think that nationalism is our future. I think looking further to the future, what we will see with regard to the individual is that, at a certain point, external entities, whether it’s corporations or whether it’s governments — they will have enough data, especially biometric data, and enough computing power to be able to understand me better than I understand myself. Very soon, Facebook or the Chinese government will be able to do that. And once you have an external entity, an algorithm, that knows me better than I know myself, this is the real turning point. This is the point when individualism, as we’ve known it, doesn’t make any sense — when democracy and the free market become completely obsolete. And we need fundamentally different models for running the world and for understanding what’s happening.

      Biometric data and personalization

      Kyle: Right. For now, several hundred years, the market as a mechanism for saying what our opinions or our desires really are, has been the most efficient mechanism. We could best allocate production towards things that people find valuable because they’re voting with their dollars. But if you can accurately say, based on this person’s heart rate, what they’re paying attention to, how they react to particular inputs, you know, whether it’s an advertisement, or some new way of interacting with things based on new technologies like VR — you could know, like, the closest thing to the underlying motivation, desire — even better than the person themselves maybe would. But at the other side of it, there’s an example you give —  and this goes back to the topic of, like, free will and individualism — lab rats that have electrodes hooked up to the reward centers of their brain. Where you have them navigate a maze, or climb ladders, and go down little chutes by basically stimulating their reward center. And it basically influences that rat’s desire. It doesn’t feel like it’s being coerced into doing that activity. It’s like…

      Yuval: Yeah, the rat doesn’t know.

      Kyle: “Oh, wow. I’m really into the idea of climbing this ladder right now. This is awesome.”

      Sonal: The rat race.

      Kyle: So, what’s interesting is, markets, as efficient as they are, like — part of how they worked was this idea of marketing to instill desires. Car ads giving you this vision of being on the open road and free, and wind blowing in your hair, and then, at some point, the desire pops up at a time when you could act on it. You buy a car. Whereas the future state that you describe is, imagine you had a headset that was like a miniaturized fMRI that can detect exactly where the parts of your brain would need to be stimulated to make you really want to play the piano right now, so that you’ll be motivated intrinsically to learn it. You could basically sell the idea of being into this. And so, being able to read your desires — but also being able to shape your desires — what do you think the interaction of those two look like?

      Yuval: We don’t know. I mean, the basic tendency is to think in 20th-century terms, that they’ll try to manipulate us. And this is certainly a danger but, intellectually, it’s the less interesting option — that, okay, they’ll use it to advertise in a different way, to shape our desires without even our knowing it, which they’ve been trying to do for decades. They’ll have better tools [for] shaping our desires. The deeper and more interesting question is, what if Big Brother can really do a better job than the individual in understanding what you want and what you need? Because many people discover during their life that they don’t really know what they want, and they often make terrible decisions in the most important decisions of their lives — what to study, where to work, whom to date, whom to marry. What happens if you have an external entity that makes these decisions for you better than you can?

      It starts with very simple things, like choosing which book to buy. If the Amazon algorithm really picks books that you are very happy with, then you’ll gradually shift the authority to choose the books to Amazon. And this may happen with more and more fields in your life. And the really interesting question is not if they try to manipulate you. The really interesting question: what if it works?

      Sonal: Oh, that’s such an interesting question.

      Yuval: What does it mean to be a human being, when all the decisions in your life are taken by somebody else who really knows who you are? It’s like being a baby forever.

      Sonal: It’s already working, on some level, because you might have a million other movies out there, but you really don’t care because you only care about what’s in the Netflix catalog because you’re looking for convenience of being able to binge-watch and get it on-demand in the moment. So, it’s already reshaping that cultural landscape. I mean, it’s already happening, [to] some extent.

      Yuval: I think the big breakthrough will come with biometric data. So, for most of these algorithms, whether it’s Amazon, or Netflix, or whatever, they work mainly on the basis of data external to my body. They follow me around in space, see where I go, which places I visit. They see my likes and dislikes on Facebook, what do I buy, and so forth. But the real breakthrough will come when they start receiving more and more data from biometric sensors on or inside my body.

      Sonal: Right, like quantified cells, wearables.

      Yuval: Yeah. I read a book, and Amazon knows exactly what is the impact of every sentence I’m reading on my heartbeat, on my blood pressure, on my brain activity. This is really where you can see how an external system can make better decisions for you than you can make for yourself. 

      Kyle: Yeah, today, these systems are basically reflecting ourselves back at us. If you look at products — because of cookies, when you go elsewhere on the web it’s like, “Oh, I see that thing again.” Like, it’s just being reflected back at me. Same thing with your Netflix queue. I gave certain star ratings to certain things. It’s reflecting that same pattern back at me with recommendations.

      Something that’s interesting to me is the idea of mapping concepts in a future space using deep learning, and then basically projecting it in different forms. And so, the idea of tracking what your eyes are looking at, what’s keeping your attention, what makes your heart rate get up, what makes your eyes dilate while you’re reading a book — you can imagine, as you’re reading it, being formatted and communicated in different ways, because they know this different way will reach you better and you’ll be more receptive to it. And so it might not necessarily be what feels coercive to us — a system of plugging an electrode into your brain and saying, “Now you’re gonna care about reading history.” It’s gonna say, “Here’s the optimal way to present history to this specific individual.”

      Yuval: This is especially being explored in new educational methods. An AI teacher that studies you while it is teaching, and adapting to your particular strengths and to your particular weaknesses. Also, breaking down all the traditional limitations and barriers of modern education. Modern education takes place in school, and you have this division. There is school and there is real life outside school. And, also, in school — now, if you have — consider [if] you have a single AI mentor that follows you around everywhere…

      Sonal: Your whole life.

      Yuval: …24 hours a day, connected to biometric sensors on your body, and there is no longer any division between school and life. There is no history teacher and mathematics teacher. You have the same teacher for both. And you don’t have to be part of a group, like, the 30 other kids in the class.

      Kyle: Basically, an AI assistant where it’s constantly in Socratic debate with you.

      Yuval: Yes.

      Kyle: Kids are inclined already to say, like, “Okay, but why? Okay, but how? Okay, but why?” And they keep digging kind of deeper until you as a parent or teacher are just like, “Because it is, okay?” Whereas an AI system, assuming it’s mapped out, like, the entire cannon of human philosophy and knowledge, could basically just keep going. Even if it doesn’t go all the way to that extent, you could have a huge increase in productivity of, you know, education, just by providing those kinds of tools to kids.

      Sonal: Mass personalization. I mean, I come from the world of developmental psychology and education, and the Holy Grail has always been this idea of mass personalization, to be able to customize. But I want to make two points. One, I agree with this idea. Vygotsky had this idea as a constructivist way of learning. You’re constructing, you’re learning your world, and that’s how you learn these concepts in a very fundamental way. And it’s really ironic, because educators have been trying to fake that in the school setting for years — by Montessori methods and all these other — Reggio Emilia — because of this false artificial divide between real life and school. The flip side, however, and I don’t think we can ignore this, is that there is a social element to why school matters — a socialization component that has arguably nothing to do with education — and where there is shared learning and collaboration and the interaction of students. And so, I wonder what this means for that.

      Yuval: You can have it outside school as well.

      Sonal: You’re saying there’s no distinction between school anymore. It doesn’t matter.

      Yuval: It doesn’t have to be limited — that all your friends are the same age as you. There is no reason why the group with which you socialize in school, everybody has to be the same age.

      Sonal: Well, that actually is another way that technology brings you back to the past, because if you think of “Little House on the Prairie,” the schoolhouse was essentially all the grades in a single school because of physical location. But you’re arguing that those boundaries, the idea of a schoolhouse, essentially melts away.

      Kyle: That feels like an inevitable transition anyway, whether it’s corporations or education. It’s this idea of, “take in this large set of inputs, crank out some modified set of outputs that fulfills some need.”

      A new technology-based ethics

      Sonal: Well, the question that I have for you guys, and especially given “Sapiens” and the theme of “Homo Deus,” is what do humans have to believe in order to make this reality continue happening? Do they not have any agency in any of this? Because it sounds like we’re almost talking about, you know, these uploaded brains in a vat. Is there any sense of coordination, consciously? Is there a new religion? I used to watch “Star Wars” as a kid. I remember thinking to myself, because I grew up Hindu — and you learn a lot about all these Hindu gods and goddesses. I remember thinking, this reminds me a lot of hearing about the Mahabharata and all these other things happening. Anyway, I would argue that science fiction is like religion for a lot of people, but what do people have to believe in this new world? What is their religion? Is there one? I mean, you make the argument about — data is a new religion, but that sounds, to me, more of something that’s there versus something that people are choosing, like, creating new myths and gods around actively.

      Yuval: I think we are seeing, and we will see more and more, the rise of kind of techno religions. Religions based on technology that make all the old promises of traditional religions — they promise justice, and happiness, and even immortality in paradise. But here on Earth, with the help of technology, there already has been one very important techno religion in history, which is socialism.

      Sonal: Oh, I never thought of that that way.

      Yuval: Which came in the 19th century with the Industrial Revolution. And what Marx and Lenin basically said — “We will create paradise on Earth with the help of technology,” steam engines and electricity, and so forth. When Lenin was once asked to define communism in a single sentence, the answer he gave was, communism is power to the workers’ councils, plus electrification of the whole country. You cannot establish a communist regime without industrialization. It’s based on the technology of the Industrial Revolution — electricity and steam engines, and so forth. And the idea is, we’ll use this technology to create paradise on Earth. It didn’t really work very well. So, now, I think we will see the second wave of techno religions. Now we have genetics, and now we have big data, and, above all, we have algorithms. They’re our salvation. Paradise will come from the algorithms.

      Kyle: You talk about, in the book, the idea that the more you commit or sacrifice on behalf of your ideology or religion, the more you buy into it — because you have this sunk cost. And so the idea of, like, sacrificing a goat or a cow to a god made you buy more into, because I can’t have spent the last eight seasons sacrificing goats and have it been for nothing. So, looking forward then, we’re hitting some kind of productivity cap as normal humans — that autonomous machines and systems are going to beat us, so we have to sacrifice our own humanity to increase our own productivity and augment ourselves. You can also see the emergence of some kind of powerful ideology. Like, the religion of the 21st century onward is, we are the gods?

      Yuval: This is actually an old idea. Humanism, which goes back to the 18th century, even 17th century, is saying humans are the gods. Humans are the source of all meaning and authority. Everything you expected previously from the gods — to give legitimacy to political systems to make decisions in ethics. Humanism comes and says the highest source of authority in politics is the voter. The highest source of authority in economics is the customer. The highest source of authority in ethics is your own feelings. Humans are the gods.

      Now we are entering a post-humanist era. Authority is shifting away from humans. If, in the last 300 years, we saw authority descending from the clouds to Earth to humans, now authority is shifting back to the clouds — but not to God, but to the Google cloud, to the Microsoft cloud. The basic idea of this — if you want [a] new religion or new ideology, is again — if given enough data and enough computing power, an algorithm can understand me better than I understand myself, and make decisions for me. In the end, religion is about authority. The basic question of religion: where does authority come from? And the answer of the 21st century: authority doesn’t come from humans, authority comes from data and from data processing. There is also an underlying new ontology. What is the world? What is reality? In the end, reality is just a flow of data. Physics, biology, economics — it’s all just a flow of data.

      Sonal: It’s all a type of algorithm.

      Kyle: We are just computers interpreting some fraction of reality.

      The future of work and UBI

      Sonal: They’re all algorithms. That’s the connective tissue of everything, from biology, to computers, to everything. I have a quick question for you here. What does this mean for the future of the firm — work? I would love to hear your thoughts on the universal basic income debate that’s playing out around the world right now, because that’s essentially people opting out of the rat race, in some arguments.

      Yuval: I think we need new economic models in place. For the moment, when AI and robots, and so forth, may push more and more humans out of the job market. And we might see the creation of a new class of people who are not just unemployed but unemployable. At present, the best idea so far that people managed to come up with is universal basic income. The problem there, is that we don’t really know what universal means, and we don’t really know what basic means.

      Sonal: Right, and where the income comes from, but that’s another sidebar.

      Yuval: No, let’s say you tax and use the proceeds to give people universal basic income. Now, then the question is what is universal? Would we see the day when the U.S. government taxes the profits of Google in the U.S. and uses it to pay people in Bangladesh or Mexico who lost their jobs? So this is the first issue of universal, because now the economy is global, and a disruption of the economy, say, by the rise of AI will require a global solution. And most people who think about universal basic income, they think in national terms. Universal, for them, means U.S. citizens. The other problem is, what is basic? Basic human needs keep changing all the time. We are beyond the point when basic needs meant food and shelter.

      Kyle: And the problem is that humans are biased towards looking at examples that are based on who you know. It’s hard to see, kind of, that level of UBI pulling it off. It feels like people’s expectations would be much higher depending on where they are and what life they’ve already lived.

      Yuval: The basic problem is that people’s expectations keep changing. Usually, they grow. As conditions improve, expectations increase. And, therefore, what you see is that even though the conditions over the last centuries of most humans have improved quite dramatically, people are not becoming more satisfied, because their expectations also increase. And this is going to continue in the 21st century.

      Sonal: Yeah, I have a question here, because in “Sapiens,” you said something that I thought was very profound when I read it, which is that the agricultural revolution was actually one of the greatest frauds ever perpetrated on ourselves. And so if you think about this shift, from Agricultural Revolution to Industrial Revolution to now, essentially, Information Revolution — what’s the fraud that we’re perpetrating on ourselves now? Where does meaning come from? Because I think the thing that people often forget to address when they talk about the universal basic income and, you know, future of work debate is this idea of meaning. And does that even matter at the individual level?

      Kyle: Restless people tend to pick up the pitchforks.

      Sonal: Right, exactly. Exactly, because it also goes to your points — and this is a universal theme that we have to address on some level — of further entrenching inequalities. That’s an important thing to think about.

      Yuval: There are two different problems. I mean, first, you have inequality. And once more and more people no longer work, they depend on, say, universal basic income, then they have no way of closing the gaps. They depend on charity, on whatever the government is able or willing to give them, and you just don’t see any way in which they can close the gap.

      Sonal: That’s if they’re dependent on it, because it can also be something that’s supplementary to something else you do.

      Yuval: I’m thinking in terms of what happens if, again, AI pushes more and more humans out of the job market, so they rely on universal basic income. And it provides whatever it provides, but if they want more, they just have no way of getting more. So this, kind of, entrenches inequality. And if you add to that biotechnology and bioengineering, you get for the first time in history the potential of translating economic inequality into biological inequality.

      Sonal: Yes.

      Yuval: If you look back at history, let’s say, the Hindu caste system — people imagined that the Brahmins are superior, they are smarter, they are more creative, they are more ethical. But, at least as far as scientists today are concerned, this wasn’t true. It was all imagination.

      Sonal: Right. It was not true at all.

      Yuval: It was not true. It wasn’t true that the son of the Brahmin or the son of the king was biologically more capable, smarter, more creative, whatever, than the son or daughter of a simple peasant. However, in the 21st century, it might be possible for the first time to translate economic inequality into real biological inequality. And once this starts, it becomes almost impossible to close the gap. So this is one problem of a rise in inequality. Another problem is the question of meaning — that even if you can provide people with food, and shelter, and medicine, and so forth, how will they find meaning in life? For many people, their work, their jobs provide them with meaning. “I’m doing something important in life.”

      Sonal: A mission. “I believe in this.”

      Yuval: Yeah. So, one of the answers, some experts say, is that people will just play games most of the day. They’ll spend more and more time in virtual realities that will provide them with more meaning and more excitement and emotional engagement than anything in the real reality outside.

      Kyle: Everyone just lives in their perfectly-optimized-for-them Holodeck.

      Yuval: Exactly.

      Sonal: Because you’re freed from the constraints of the physical realities.

      Yuval: Yeah, and you get your meaning from the game, from the virtual reality game. And in a way, you can say, oh, this is nothing new. It’s been happening for thousands of years. It’s simply been called religion. I mean, religion is a virtual reality game that provides people with meaning by imposing imaginary rules on an objective reality. You play this game [where] you have to collect points. If I eat non-kosher food, I lose points. And if by the time I die, I gathered enough points, then I go up to the next level.

      Sonal: I mean, in Hinduism, karma is essentially this great game of collecting and subtracting points across multiple lifetimes.

      Yuval: Exactly.

      Kyle: So, really quickly — this goes back to the automation, kind of, question and potential future. If you look back at, kind of, the Industrial Revolution, where humans as mechanical actors — just imbuing something with value by acting on it with their hands, or bodies with agriculture — that became less important as using animals, and then machines, were able to do that same task much more efficiently. Now, humans are valuable because they are knowledgeable operators of that machine. As part of the Industrial Revolution, the shift to services led to this idea that we’re not just investing in capital, we’re investing in human capital. We’re making people smarter so that they’re better at their jobs. Now, with AI systems, suddenly, again, you can just kind of buy knowledge capital as this thing that can be dropped in. Okay. An argument I hear…

      Sonal: AI as a service, even.

      Kyle: Right, how humans remain valuable is, well, we’re still social animals. We still are better than any machine at interpreting how other people are thinking about this and, you know, assuaging fears, or whatever it is — where the power of empathy is what humans will bring to the table. An interesting point you make is, actually, how humans accomplish a task — a doctor giving bad news about a cancer diagnosis. They are looking at the physical way that a person is moving their facial muscles, how their tone changes, how their voice cracks as they feel a certain emotion.

      And if you look, that’s actually just pattern recognition, which is exactly what deep learning is good at. And so, is that even an advantage humans are gonna have, or are computers gonna be much better at looking not just at those exact same features that humans can, but also, like, zooming in on the eyes and looking at dilated pupils, and guessing at heart rate by looking at someone’s wrist or chest? What are humans going to be good at? What should people be investing in for, you know, the future to come?

      Sonal: Yeah, what happens when human capital becomes commodified?

      Yuval: We don’t really have an answer. Yes, many people, when they reach that point, they say, “Okay. We’ll invest in social skills, in empathy, in recognizing emotions.” The emotions are like the last…

      Sonal: Emotional intelligence.

      Yuval: The last frontier. But the thing is that emotions are not some spiritual phenomenon that God gave homo sapiens to write poetry.

      Sonal: Another electrochemical, just like everything else.

      Yuval: Emotions are a biochemical phenomenon. There are biological patterns just like cancer. When your doctor wants to know how you feel, he or she basically recognizes patterns in two kinds of data, as you mentioned. It’s what you say, and, actually, the tone of your voice — even more important than the content of what you’re saying. And, secondly, your body language and your facial expression. When my doctor looks at me at the clinic, she doesn’t know what’s the level of my blood pressure at the moment. She doesn’t know which parts of my brain are activated right now. 

      But an AI, potentially, will be able to know that in real-time using biometric sensors. It will have much better sources of data coming from within your body. So their ability to diagnose emotions will be better than the ability of most, if not all, humans. So what will humans do? We don’t know. Nobody really has an idea, a good idea, of how the job market would look like in 30 or 40 years. We’ll have some new jobs. Maybe not enough to compensate for all the losses, but there will be new jobs. Problem is, we don’t know what they are. Because the pace of change is accelerating, it’s very likely that you will have to reinvent yourself again and again and again during your lifetime if you want to stay in the game.

      Sonal: Right, when you don’t have premature death anymore, and you live your full life, or you even have extended longevity through technology, you can reinvent yourself, like, 10 times until you’re 100.

      Yuval: The basic idea for thousands of years was that human life is divided into two periods. In the first period of life, you mostly learn. You learn skills, you gain knowledge. And then, in the second part of your life, you mostly work, and you make use of what you learned earlier. This is now — it’s going to break down. By the time you’re 50, what you learned as a teenager is mostly irrelevant.

      Sonal: It’s already true right now.

      Questions for the future

      Kyle: So, now, you know, again, thinking about autonomy — you know, we’re already seeing the shift towards smaller militaries with really advanced equipment and fighter jets. And we’re gonna see robots on the battlefield. As humans become less valuable economic actors, as they become less necessary to fight for power at, kind of, that scale, how does that factor into, you know, the extension or lack thereof of, you know, political agency?

      Yuval: Most people today have absolutely no military value. In the 20th century, the most advanced armies relied on recruiting millions of common soldiers to fight in the trenches. Now they rely increasingly on small numbers of highly professional soldiers, super-warriors, all the special forces and so forth.

      Sonal: Surgically targeted.

      Yuval: And they rely increasingly on sophisticated and autonomous technology, like drones, and robots, and cyber warfare. So you just don’t need people militarily as before, which means not only that they are in danger of losing their political power, but also that the government will have a far smaller incentive investing in their health, and education, and welfare. Maybe the biggest project and achievement of most states in the 20th century was to build these massive systems of education, and health, and welfare.

      Sonal: Safety nets.

      Yuval: And you see this not only in democracies but also in totalitarian regimes. But if you don’t need them as soldiers or workers, then the incentive to build hospitals, and schools, and so forth diminishes. In a country like, I don’t know, Sweden, I think the traditions of the welfare state and the social democracy will be strong enough that the Swedish state will continue to invest in the education and health of most of the people there, even if there is no military or economic necessity. But if you think about large developing countries, it’s much, much more complicated. If the government doesn’t need tens of millions of Nigerians to serve as soldiers and workers, maybe it will not have enough incentive to invest in their health and education. And this is very, very bad news for most of the human race, which lives in places like Nigeria and not in places like Sweden.

      Kyle: And so what’s the best course of action to follow if that’s the case? Is it, make sure that the most inclusive institutions possible are in place before that transition happens or…

      Yuval: We don’t have enough time. I think that we are not talking in terms of centuries. We are talking in terms of decades, and once the transition takes place, especially in the civilian economy. In the military, it already happened. We are there. In the civilian economy, maybe we have 20 years, 30 years, 40 years. Nobody really knows. It’s a very short time. If we don’t have a workable model by the time the transition is in high gear, then it’s going to be both an extremely difficult situation for the majority of people in the world, and the social and political implications are going to destabilize the whole world, including the first world.

      Sonal: You walked in your book, your new book, a lot about how there are three types of capital that — raw materials and energy, but people have ignored a third type, which is knowledge. And my question, from just an economic perspective, is how does this tie into how we think about growth? Especially given what you just talked about — this need to enlarge the pie in order to avoid war and violence.

      Yuval: It’s often thought that there is a limit to the potential growth of the economy, because there is a limit to the amount of energy and raw material we have access to. But this is, I think, the wrong approach. We have a third kind of asset, which is knowledge. And the more knowledge you have, the more energy and raw materials you also have, because you discover new sources of energy and new sources of raw materials. I don’t think that we are going to bump into a limit in terms of, “Oh, there is not enough oil in the world. There is not enough coal in the world.” This is not the problem. The problem is probably going to come from the direction of climate change and ecological degradation, which is something very different. People tend to confuse the two problems — not enough raw materials and the problem of climate change — but they are very different.

      Sonal: I actually wanted to probe about this, because in “Sapiens,” one of the things that you talked about was how we’ve had waves of climate change throughout the entire history of our planet. And I’m no climate change denier by any means, but I can’t help but ask the question, you know — whether we are the cause or it’s a cyclical effect — what it means for what the next cycle of change will be. Because the one thing that came through loud and clear was how every wave of climate change has brought about a corresponding change in human evolution.

      Yuval: Well, there certainly have been many periods of climate change before, but it does seem that this time it’s different, that this time it is caused, to at least a certain degree, by human action and not by some natural forces, like plate tectonics or ice ages or things like that. And the potential impact for human civilization and for most other organisms on the planet is catastrophic. So, you know, it could be both natural causes and human causes at the same time. It doesn’t make it any better. It just makes it worse.

      Sonal: The effects are the effects, right. In your book, you have this beautiful quote, which I thought was really straight articulation. “Modernity is a deal. All of us sign up to this deal on the day we were born, and it regulates our lives until the day we die. Very few of us can ever rescind or transcend this deal. It shapes our food, our jobs, and our dreams, and it decides where we dwell, whom we love, and how we pass away.” And I wanna know if you have any parting thoughts for people whose lives are being shifted by some of the technological change?

      Yuval: Since the main theme has been technology and the future of technology and its impact on society and politics, I think that my closing thought is that technology is never deterministic. You can build very different kinds of political and social systems with the same kind of technology. You could use, you know, the technology of the Industrial Revolution — the trains, and electricity, and radio — you could use them to build a communist dictatorship, or a fascist regime, or a liberal democracy. The trains did not tell you what to do with them. In the same way, in the 21st century, we’ll have artificial intelligence and bioengineering, and so forth, but they don’t determine a single outcome. We can use it to build very different kinds of societies. We can’t just stop technological progress. It won’t happen.

      Sonal: It’s inevitable.

      Yuval: But we still have a lot of influence over the direction it is taking. So if there are some future scenarios that you don’t like, you can still do something about it.

      Sonal: Yeah. Well, thank you so much for joining the “a16z Podcast.” If people have not already read “Sapiens,” they must read that, and especially the new book, “Homo Deus: A Brief History of Tomorrow.”

      Kyle: Thanks for coming in. We really appreciate your time.

      Yuval: Thank you.

      • Yuval Harari

      • Kyle Russell

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Business of Creativity — Pixar CFO, IPO, and Beyond

      Sonal Chokshi and Lawrence Levy

      You’ve heard a version of this story before: Steve Jobs calls some executive out of the blue to come work for him. Only this time the story turns out great … and the company wasn’t Apple. This episode of the a16z Podcast shares some of the journey that former CFO Lawrence Levy went on with Steve Jobs as they took Pixar — a company then on the verge of failure — to its IPO and subsequent greatest hits.

      It’s sort of an adventure story but is really more of a quest for product-market fit. How did they figure out a model for such an old-but-new business (i.e., animation and entertainment)? How did they take an improbable plan and figure out how to make it work — both qualitatively and quantitatively? How did they then navigate and straddle the diverse worlds of Silicon Valley, Hollywood, and Wall Street? And finally, how did they price their IPO, which was also a symbol of Steve Jobs’ comeback story … a narrative that’s sometimes lost in the Apple story.

      From the business of creativity to corporate culture, Levy — former CFO of Pixar, board member, and author of the new book To Pixar and Beyond: My Unlikely Journey with Steve Jobs to Make Entertainment History — shares his (and Jobs’ untold) story. But it isn’t just a story about finding the right model and numbers to build, explain, and measure the business; it’s also, partly, about how to get the measure of one’s humanity, too.

      Show Notes

      • How Lawrence Levy came to Pixar [0:48]
      • Why animation is difficult [6:00], and how they decided on the company’s focus [7:51]
      • What it was like to work with Steve Jobs [11:40]
      • Getting investors to accept Pixar [15:00], IPO pricing [17:50], and lessons learned [20:02]
      • Levy’s and Jobs’ long-term plans for Pixar [21:32]
      • Personal and philosophical takeaways from the experience [25:00]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today we have, as our special guest, Lawrence Levy, the former CFO of Pixar, who also took the company public. He’s on the board of directors of Pixar and was in the office of the president, and has written a book, just out, called “To Pixar and Beyond: My Unlikely Journey with Steve Jobs to Make Entertainment History”. In this conversation, we cover everything from his partnership with Steve Jobs, to how they figured out what kind of business Pixar should be, to the delicate balance between finance, strategy, and the business of creativity. We even cover details about how they priced their IPO, and end on a more personal note about bringing a more humanistic approach to business and corporate culture. But first, we begin with a story of how this lawyer came to Pixar. Let’s just start jumping right into your story, which essentially you describe as an adventure story.

      The early days of Pixar

      Lawrence: Yeah, I think my experience at Pixar could definitely be characterized as an adventure. It started in 1994. Imagine that I’m sitting in my office at the company I was at then, which was Electronics for Imaging. And I was sitting at my desk and the phone rings. I pick up the phone, and on the other end of the phone, I hear this voice that says, “Hi, this is Steve Jobs. I saw your picture in a magazine one day.” He said, “I thought we would work together someday. And I have a company that I would like to tell you about.” And I immediately thought that he was talking about NeXT Computer, because NeXT was very well known. It had gone through quite a number of bumps. But then he says, “It’s Pixar.”

      Sonal: Had you even heard of it?

      Lawrence: I had heard of Pixar, but I didn’t know anything about it.

      Sonal: How jarring because you’re hearing from this computer founder who’s suddenly going to a cartoon company. Or how would people have described Pixar at that point?

      Lawrence: Oh, I wouldn’t have known enough to even know it was a cartoon company. I didn’t even hardly know what to say. But I was intrigued. It was just Steve Jobs on the end of the phone. He’s telling me about something called Pixar.

      Sonal: So at that time, Pixar was a flailing company. I mean, it wasn’t at all the success that we know today. It was very early days. What were they? Were they a movie company, an animation company, an entertainment company?

      Lawrence: If you did research at that time, which I did, you would have discovered they were really a graphics company. So, Pixar had really set out to change the world of high-end computer graphics. And they made an imaging computer. They’d made some short films that had, sort of, won accolades at some of the festivals and shows, but not even Pixar conceived of itself as an entertainment company in 1994.

      Sonal: It wasn’t an entertainment company at the time. Clearly, it became one.

      Lawrence: It was in a broad range of businesses. It made software, it did animated short films, it did commercials. It had this little project going on called “Toy Story.” And so my goal, my work, was to, sort of, figure out how to put all that together. And at the beginning, the last thing that I imagined was that it was gonna be an entertainment company. Why would you hire me to go in there and figure it out? Because I… 

      Sonal: What was your background?

      Lawrence: I was a lawyer and an executive. I was at the Wilson Sonsini law firm. I represented all these high-tech startups, I knew software, I knew hardware, I knew semiconductors, but I didn’t know any more about entertainment than, you know — that you go to a movie theater, buy your ticket, and buy some popcorn. So, it never occurred to me that I was going to run an entertainment company. And I actually thought it was going to be a company where we would pull together different businesses, and they would help — each of those businesses would — the risks of one would offset the other.

      Sonal: It’s interesting you say that, because one of the lines that really stood out to me is someone telling you, think of it as a portfolio business. And I think that’s really interesting, because it actually touches upon all kinds of businesses. If you think about an R&D of a big Fortune 500 company, that’s having a portfolio framework for deciding which projects to invest in and not to invest in. If you think of VC, that’s a portfolio business. How does that portfolio mindset apply specifically to Pixar?

      Lawrence: Well, at the time, I thought that it could be a business where its different parts would offset the risks of the other. And the reason was because the notion of becoming a standalone, independent animated feature film company was a crazy idea. 

      Sonal: Why was it so crazy?

      Lawrence: Because only one company in history had ever done it.

      Sonal: Disney.

      Lawrence: That was Disney. And they had only succeeded at it for a few years, from 1939, when they released “Snow White and the Seven Dwarfs.” They did “Dumbo” and “Bambi” and a few of these great films. But it was such a struggle to be a standalone, animated feature film company that Walt Disney — brilliantly — began to diversify. And so then you get the theme parks in 1954, in 1955, and they went into ABC television, and the Wonderful World of TV, and they opened Buena Vista distribution. They couldn’t sustain themselves as an animated feature film company. If you looked at the history, every other studio that had tried to do that had essentially failed. So, the notion that you could build an independent, standalone, animated feature film company was one that I basically fought every step of the way, because it seemed impossible.

      Sonal: Well, there’s actually an interesting exchange between you guys. Steve told you, “Investors will love this. Pixar can be in a multimillion-dollar video market.” And you responded, “I agree, but let’s get the data first. Even then I’m not sure we can count on home video alone to take Pixar public.” So, tell me about that. How did home video play a role in all of that?

      Lawrence: Well, the video market was changing the nature of the film business. That was the time when Blockbuster films and Blockbuster stores and — it was creating a whole new economy for film, which was great. And Disney was having incredible success at it. But the challenge for Pixar was that animated feature films take a long time to make. So, if you release a film every four years, that’s too much time. And if you have one film that’s a miss, which could easily happen, you could have an 8 or 12-year dry spell, and the home video market wouldn’t be any help at all.

      Sonal: That’s kind of frightening, actually, to think about because you can now pivot a company three times and not even blink an eye, and here your product is this thing that has so much planning and development that goes into it. Just to break it down at the technological level, why is animation so hard?

      Lawrence: An animated feature film has about 110,000 frames in it. And each one of those frames has an enormous amount of data. At that time, just to render a single frame, could take a day. So imagine how much computing power it would take to render an entire film. And then you had to have the software to actually manipulate the characters — a three-dimensional image of Woody or Buzz or something like that. How do you then manipulate that image to make it seem as though it’s alive, to give it emotion? Because emotion is very subtle movements of the eyes, very subtle movements of the mouth. Pixar had to invent the technology in order to do that.

      Sonal: So, given this context of the difficulty of the animation itself, the fact that you can only have maybe one good movie every four years, and you’re putting a lot into one movie, how did you guys get to “Toy Story?”

      Lawrence: Pixar was already making Toy Story when I arrived. They had committed to that in a 1991 contract that they went into with Disney. That was a little bit of a Hail Mary, in a way. Because otherwise, Pixar was perilously close to going out of business. Disney came along and said that they would, basically, front the costs for making that first animated feature film. But the price that Pixar paid for that was enormous. It was almost like selling your soul.

      Sonal: Why?

      Lawrence: Because the contract — and this was really my hardest moment. You know I was a lawyer? This shows you how little I knew then about the entertainment business. They had entered an agreement with Disney, but it was written in this arcane Hollywood code. When I got there, I realized that Pixar had essentially been tied up by Disney for what could have been, you know, 12 years, 15 years with a very, very tiny share of the profits.

      Sonal: Oh, that’s awful.

      Lawrence: And that was only if the films were ridiculous blockbusters. It just didn’t seem to have any chance. Its hands had been tied too much.

      Developing the company’s focus

      Sonal: So how did you guys come out of that? I mean, how did you go from this situation to the success that you and Steve and everyone that worked at Pixar built? 

      Lawrence: Well, we went off on this quest, if you will.

      Sonal: This is an adventure story.

      Lawrence: It’s an adventure. So there are two parts to it. One is qualitative. What does it mean to build an entertainment company? And the other is quantitative. What do the numbers look like? At the time, we didn’t even have a spreadsheet of numbers that told us how these movies performed. I recount the story in the book of how we had to beg, borrow, and steal, just to understand how the numbers were gonna work. We’re starting to very slowly understand how these films make money, basically. And at the same time, Steve and I are shuttling back and forth to Hollywood. And we’re meeting these, you know, Hollywood executives. We met Edgar Bronfman Jr. and we met Mike Ovitz, Joe Roth. Basically, anybody who would open the door to us. We’d ask them all these questions about the industry, because we thought we can’t just do straight animated feature films. So we wanted to learn about all those other businesses.

      Sonal: Was Steve on board with that diversification? Because I’ve always heard famous stories about him being such a focus guy. Did he agree that you had to diversify in order to make this a viable business?

      Lawrence: I think Steve was on board with the process that we were going through. He hired me to assess and analyze this business, and basically partner with him to do that. So, we went on this quest. We learned all about the live-action film business, but the live-action film business is also a terrible business. It actually didn’t diversify the risk of animated feature film. We came to the notion that Pixar should be an animated feature film company, basically by default, which is that this is our only shot. We literally created the spreadsheet and it had on it, this is how many films we have to release and this is how they have to perform in order for Pixar to have a shot, and these are the conditions that have to happen in order for that to work.

      Sonal: And that was a quantitative activity that you guys did.

      Lawrence: That was quantitative, but associated with qualitative notions of — this is what that means. We have to get out of the software business. We have to get out of the animated commercials business. We have to triple the size of the company. We have to increase output. We have to do all these things. We have to renegotiate with Disney. And if we do all those things, and if our films are more successful than anybody will possibly believe they could be, we could make it. And so, it was literally a million to one shot that you could pull it off. And then I was like, “I have no idea how we can get this financed,” because the risks in this plan are absurd.

      Sonal: Insane. That’s interesting. You described that Steve had hired you to partner with them. So you guys were partners?

      Lawrence: We represented the business and strategic side of the company together. The creative side of the company was represented by John Lasseter and Ed Catmull.

      Sonal: And Ed Catmull, of course. So, we have the creative side, and you have the business and strategic side.

      Lawrence: Yeah.

      Sonal: Those are essentially three legs of a stool. And they all have to be functioning in order to make the company succeed, and build this amazing thing. How did you guys make decisions, though? It’s not like you just had spreadsheets and these rubrics. Was there, like, a visionary, sort of, “We’re gonna go do this?” Was there an instinct? Was there someone who’s saying no and yes? How did you guys negotiate that?

      Lawrence: With Steve, and with Ed as well, and John, it was basically a constant dialogue. Decisions came out of this dialogue. It was like we were in motion all the time. So there weren’t, like, moments where we’d sit down, and now we have to decide. It was more like it emerged from this continuous dialogue.

      Sonal: It sounds like an incredibly creative friction.

      Lawrence: It is. It is, because we never fought, but we didn’t always agree. There was this healthy debate, I would call it, one pushing the other, always pushing the other, you know, to sort of, figure it out.

      Sonal: Never fought but didn’t always agree. That is the definition of healthy conflict.

      Lawrence: Yeah, that’s how I would describe it. It was collaborative and respectful and great.

      Working with Steve Jobs

      Sonal: So, tell us some of your stories about partnering with Steve Jobs. It’s a really interesting moment right now, in recent history, given how few years it was that he passed away. But already three or four books have come out trying to repaint the picture, or maybe paint the picture. And you had a unique, front row — not just a seat, but you were an active participant.

      Lawrence: One of the reasons I was motivated to write the book was, in the aftermath of Steve’s tragic death, all these things came out. And I started to see, “Well, what about the Pixar story?” I mean, it was a little bit of an afterthought. Some of it made it seem as though, you know, he started at Apple, and then he went away from Apple, and then he went back to Apple. And I’m like, “Well, wait a minute, that part when he went away from Apple — that was really important.”

      Sonal: That was a really big deal.

      Lawrence: Both as Pixar and in Steve’s life, it was a really big deal. Steve had had a series of failures leading up to Pixar. Before he left Apple, there was the Apple Lisa, and there was the Apple Macintosh. After he left Apple, there was the Pixar imaging computer. And then there was the NeXT Computer. So, those were four pieces of hardware that essentially failed in the marketplace. Pixar really was his comeback, as I recount in the story. But Steve and I hit it off from the get-go.

      Sonal: Why do you think you guys hit it off? I mean, everyone’s heard that famous story in the Apple counterpart, where he’s saying, like, “Do you want to sell sugar water the rest of your life?” And this is, sort of, like, a similar thing where it could have gone that way, where there might not have been as much chemistry between you and him. What made you work?

      Lawrence: That’s a really good question. Sometimes there’s just chemistry. And from the moment we met, there was just this level of mutual respect and trust, and it lasted for a long time. And it was both professional and personal. We just kind of got each other.

      Sonal: That’s amazing.

      Lawrence: And so the relationship started in this, sort of, you know, adventurous dialogue of this crazy thing called Pixar. Of course, I’m aware of all the other accounts of Steve. I’m writing about my experience.

      Sonal: You had the opportunity to work with, arguably, one of our most influential innovators, who’s created things that have changed our world. It’s kind of an amazing opportunity. What were the moments that were most trying between you guys, because, obviously, it wasn’t all peaches and cream?

      Lawrence: I think when Steve began to see that an IPO was possible for Pixar, he couldn’t get there fast enough.

      Sonal: He couldn’t get there fast enough. Interesting.

      Lawrence: He couldn’t get there fast enough, because that IPO was really the symbol of his comeback. He was rushing toward that faster than I was.

      Sonal: Which is, by the way, really interesting, because in the current ecosystem, it’s been, sort of, the opposite, where a lot of founders are not ready to IPO yet. One of the things you said in the book is that, “Besides making films that would enjoy unprecedented box office success the world over, we simply had to, one, quadruple our share of the profits, two, raise at least $75 million to pay for our production costs, three, make films far more often than we knew how, and four, build Pixar into a worldwide brand.” Piece of cake. But really tough. That was your plan.

      Lawrence: Yeah, that was the plan.

      Sonal: So how did you guys go from there? Because the thing that I think is super interesting is that you said it’s not going to be easy for investors to get their heads around Pixar’s business. We have a lot of explaining to do. Tell me about that.

      Investors and IPO

      Lawrence: Well, I went to see my old friend and mentor Larry Sonsini, of Wilson Sonsini. And I presented all this to him. And he knew Steve really well, and he knew Pixar. And I said, “Investors are really gonna balk at this. We’ve got to disclose this — all this risk, up front.”

      Sonal: Yeah, it’s a lot of risk.

      Lawrence: So, I thought he was gonna say, “Forget about it. You have no chance. Don’t even think about it.” But he didn’t. He said, “You’re right. This is an incredible long shot. But if you disclose the risks upfront, completely, then you’ll see investors will make an open evaluation.” And that’s actually always been my approach to, you know, when I was in business, and being a CFO, which is — don’t hide behind the risks. You know, just put it out there. And don’t be afraid to put it out there. Because they’re going to find out anyway.

      Sonal: And they’ll probably trust you more, actually, for being open about it.

      Lawrence: And they’ll trust you more, so just be an open book. Because once you’re a public company, you’re gonna be in a fishbowl anyway. I said to Steve, “You know, we’re gonna have to disclose this risk, and these things that we have to accomplish that are so difficult, and why they’re so difficult, upfront.” Steve wasn’t against it, but he was kind of like, “Fine, fine. If that’s what we need to do, we’ll do it. But the investors won’t care about that, because they’ll see this incredible vision and this incredible possibility.” So, I had one foot on the brake while he was rushing toward it. We were heading into some very choppy investment waters, which we did.

      Sonal: How did you navigate that, exactly? Because you have to also have a model that works for people to want to invest in you. One of the things I’ve heard is that — a big part is the story you tell on your way to the IPO. How did you guys tell that story?

      Lawrence: The story was, in some ways, the easy part. By that time, we had figured out Pixar was going to become an entertainment company. So, we told the story of its creative capability, its production capability, its technical capabilities. We laid it all out — the talent and the capacity. That part of the story was the easy part.

      Sonal: The easy part, okay.

      Lawrence: It was easy to tell. You couldn’t help but walk into Pixar — even today. But in 1995, when you walked into Pixar…

      Sonal: Before the new campus.

      Lawrence: …before the new campus, and you start looking inside a little bit, I mean, it’s staggering. There wasn’t a person there that didn’t leave just, like, blown away. It’s that impressive.

      Sonal: That sounds amazing.

      Lawrence: It’s really impressive. So, that was the easy part. It was the numbers part. But we did have a model. And we said if these things happen…

      Sonal: Those four things.

      Lawrence: Right. We build a worldwide brand, we quadruple our share of the profits, we raise $75 million, we release films more often — if we do all those things, this company will make it, and it’s really up to the investors to assign a risk to that. You may think we have a 1% shot, or we have a 10% shot. That’s for you to decide. But if we hit those, then we will hit the ball out of the park. It was more than a prayer.

      Sonal: Well, this is where the pricing of the IPO is a really fascinating narrative. Because you have a version of the story where it’s like, “Okay, if we do these things, these four things, we will hit these amazing numbers.” The investors are like, “Okay, yeah, that’s a great story, and great model and plan. And yay, we’ll give you capital to do that. But hey, we’re not gonna value very highly upfront that we believe in this.”

      Lawrence: Well, I would say that the price of an IPO on the financial side of a company is one of the most important decisions that it will make. And this was also a point of contention between Steve and I, about how to price it. My thinking about it was that it’s important to leave something on the table. It’s much more important that early investors be happy, and feel like they made money, than they’d be disappointed and feel like they lost money. And so if you underprice a little bit and they feel like they’ve made money, then you get a lot of confidence in your stock going forward.

      Steve felt that, you know, wherever we priced it, it would just skyrocket. And the irony — sitting here doing the a16z — is that the reason he thought that was because of Netscape. Netscape and Pixar were the two hottest IPOs of that year, in 1995, but Netscape went first. And they, of course, were the first company to ride this new wave called the internet. It was huge. I’m looking at Pixar saying that, you know, Pixar is amazing. But no one is sitting around talking about, you know, the animated film business. This is a 50-year-old business.

      Sonal: And the best model, till that point, was 1939 founded company of Disney.

      Lawrence: Yeah, Steve was saying that if they’re valued there, then we ought to be valued there. And I was like, “But it’s different. There’s no frenzy out there for animation.” We went back and forth, back and forth, but we worked it out.

      Sonal: How did you guys work it out?

      Lawrence: The pricing comes together through a whole combination of factors. You have investment bankers, and they’re having a lot to say about it. You’ve got your lawyers and the disclosures, and you have your board, and there’s a lot of elements. And so it begins to coalesce. We managed to get it into a bandwidth where I felt that we could make it and investors would be happy, and Steve felt that we had a shot at the kind of valuation that he wanted.

      Sonal: I want to ask you about some lessons learned and high-level takeaways from that whole experience of going to your IPO, and things that may be different today.

      Lawrence: You know, people see their IPO as like an end game, but IPO is the beginning of the game. It’s a change event.

      Sonal: Did it make you better, because you’re able to execute on your model in the public eye?

      Lawrence: I think if you’re disciplined, that it doesn’t necessarily have to make you better. But by the time we were in public, we knew what we were doing. We really did. The plan that we’ve talked about, that we put in place, lasted Pixar 10 years. It took 10 years to execute and we just sort of…

      Sonal: Kept going.

      Lawrence: …kept going, steadfastly. Went at it.

      Sonal: How did you navigate the cultural divides, where you have the Silicon Valley-centered company, and team for that matter, technology, Hollywood, and the business of creativity — and then you have the East Coast, kind of, banking coterie? How did you navigate all those?

      Lawrence: You describe Hollywood as the business of creativity. But in some ways, one of the things we learned is that sometimes creativity in Hollywood isn’t all it’s cracked up to be. The big studios have so many presses. Some of our thinking back then is, we have to protect Pixar from some of those kinds of pressures, in order for it to continue to innovate and do original films. This is a tremendous tribute to John Lasseter and Ed Catmull, who really created the creation culture of Pixar. And Steve and myself — I think our contribution, in a way, was to recognize the importance of preserving that.

      The creative process

      Sonal: I think that is actually a pretty tremendous credit to you as well, because when you’re responsible and accountable for the financial performance, there are a lot of short-term things you can do that short-shrift some of that long-term creativity. You get eager to see some results. And to keep your eye on the long term is a very difficult challenge. All companies face this today — this balance between being able to execute on a plan, but also adapt and innovate. What are some of the ways you’ve made that work?

      Lawrence: Well, these were very big discussions. Steve and I would talk about this a lot. Because, going back to 1995, 1996, the creative team at Pixar, as brilliant as it was, was young and untested. “Toy Story” was the first film. Now these famous directors — Andrew Stanton and Pete Docter and Brad Bird, Lee Unkrich, the others from Pixar — they’d never made a film. And you’re looking at these young directors, and each film is gonna cost, say, an average of $140 million.

      Sonal: And that’s really expensive, because they’re animation.

      Lawrence: They’re animation. It’s really expensive. So you’re literally betting $140 million…

      Sonal: On this untested talent.

      Lawrence: …on this untested talent. So, the temptation to wanna go in there and make sure it’s going well is enormous, right? Because in a project of that size, slip-ups are very expensive. They come to you and say, “We made a mistake in the story,” right, you can’t fix that, like, the next day. That’s usually a $5 or $10 million mistake. So, the most important decision that we made at Pixar was basically to trust the creatives and not interfere with what they’re doing. For executives, that’s really hard for two reasons. One is that executives think they know. They think they’re creative, and they think they know. And some are, but it’s rare. And Steve and I realized that that wasn’t our domain. We could watch movies, we could critique movies, but we were amateurs and they were professionals. And so we had to trust them.

      Sonal: But what gave you — I think it’s important to push on this — the ability to trust them? They were unproven. Was there some indication that you felt like, “We can trust them?” What were the signals? Because otherwise, anyone could say, “Hey, I want to trust the creatives.”

      Lawrence: Well, it was very clear that John Lasseter was something extraordinary. And he had hand-groomed these other — sometimes we called it, like, the John Lasseter school of animation direction. But that said, even if you’re the best director in the world, you’re gonna have misses, right?

      Sonal: Yeah.

      Lawrence: He was that good, that you could place that bet. And I think you put your finger on the issue. Not everybody, in a startup company — you don’t have to have John Lasseter — but you have to be very discerning about the level of talent that you have. And I think that’s the challenge for executives. Not to think that they know, but to be able to make a real assessment about what the level of talent they have is.

      Sonal: We see this every day here in our own business, because, essentially, the definition of venture capital is you’re betting on talent. The product may or may not end up where it started off, but you’re really betting on that person. And you have to make that assessment on a combination of factors.

      Lawrence: Right. Once Steve said to me that he felt the decisions that we made at Pixar were not his, were not mine — or they were the product of this process. I think it’s very rare to find somebody to work with like that. And when you have it, one plus one makes way more than two. And that’s what our relationship was like. 

      Personal takeaways

      Sonal: Has it changed how you live your life today?

      Lawrence: Well, I went off to do something completely different. I left corporate life behind, and I wanted to explore Eastern philosophy and meditation. I love my career, I love what I did, but I felt there was something one-dimensional about it. It’s very oriented towards success and performance and acquisition, which are great, and I have no issue with any of it. But we also pay a price for having that intense orientation towards performance at all costs.

      Sonal: What’s the price?

      Lawrence: One of those prices is, it creates a stress culture. I asked myself, why is it today that we hear so many stories about anxiety and agitation and mental health? And I think a part of this is because we have this sort of performance orientation without anything else to balance it. I have all these years at Pixar. And what I learned at Pixar is that it’s all about story. Then I go off and study Buddhist philosophy. And after all those years, I realized — it’s all about story.

      Sonal: What do you mean?

      Lawrence: Pixar, of course, it’s all about story because that’s what’s driving people to enjoy the film. In Buddhist philosophy, what it’s saying is that we’re living by a story. That we can’t always see it, but our life is driven by stories that are inside of us. And so…

      Sonal: Actually, psychotherapists say the same thing.

      Lawrence: And now neuroscientists are also saying the same thing. It’s fascinating. So, the performance story that we live in now — it’s not something inherent in the nature of the universe. It’s a story. It’s a culturally-generated paradigm. The Middle Way essentially says that we are mistaking our stories for reality.

      Sonal: Why do you call it The Middle Way?

      Lawrence: Because The Middle Way talks about two extremes. One extreme would be the extreme of believing that your story is real. And the other extreme is, because that story is so real to you, you cannot conceive of any other possibility — even if the story is hurting you. 

      Sonal: Very binary.

      Lawrence: Exactly. It’s very binary. And so to move away from your story, to move away from your performance orientation, is so threatening and, sort of, fearful that you can’t imagine life is even possible over there. So The Middle Way is about, how do you find that place in the middle? It’s not about giving up performance or giving up all of these things. It’s about harmonizing with all the different elements of life.

      Sonal: Just one last question. On a personal level, there are so many interesting threads. How does this tie into companies at an organizational level, like, in terms of corporate culture?

      Lawrence: I think it goes to the very heart of corporate culture. If you look at what’s driving corporate culture over — I’m gonna say 300 years — it started with the Dutch East India Company. By 1402, for 200 years, the Dutch East India Company ruled the world. And the mentality — the corporate mentality going back is basically acquisition at all costs. Over the decades, what we see is, basically, a battleground between corporations that are trying to acquire at all costs, and other forces around them that are trying to get them to pay for the costs. So, we have to have environmental laws, and anti-pollution laws, and anti-child labor laws. They’re just going for it. And that creates that mentality of success at all costs within corporations. And that’s the paradigm that we’re in. I believe that we could change that paradigm to a more humanistic paradigm that values the individuals that are in cooperation.

      Sonal: I actually wonder if that story is changing right now, because the firm for the first time in 300 years is essentially being reinvented with all these new types of models, decentralized models, B Corp, social business. There’s a whole category of new things that are coming around now.

      Lawrence: I think that’s right. And I think it’s fantastic.

      Sonal: The theme to me — and this is sort of the theme of this whole podcast — is this human side to the business. It all ties together.

      Lawrence: I’ll tell you this, the number one question that I get when I give talks is, “Do I have to be a jerk to succeed?” People ask me that all the time.

      Sonal: That’s a question that comes up all the time. Do you have to be a brilliant jerk in order to be successful?

      Lawrence: Exactly. I’ll tell you my answer. My answer is no. You don’t have to be a jerk to succeed, but it’s harder, because you can be a jerk and succeed.

      Sonal: You’re, kind of, going against gravity.

      Lawrence: And so, if you want to do it the other way, it’s up to you. It means that whatever you’re doing — you have to make hard decisions, you have to fire people, reprimand people. Whatever you do, you have to remember that you’re dealing with a human being. It’s a choice, and it’s one that we can all make.

      Sonal: That’s great. Well, Lawrence, thank you for joining the “a16z Podcast.” And for everyone who wants to read more about these adventures, read the book. Thanks, Lawrence.

      Lawrence: Thank you for having me.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Lawrence Levy

      All About Microservices

      Adrian Cockcroft, Frank Chen, and Martin Casado

      “Incremental change may be good theory, but in practice you have to have a big enough stick to hit everybody with to make everything move at once”. So shares Adrian Cockcroft, who helped lead Netflix’s migration from datacenter to the cloud — and from monolithic to microservices architecture — when their streaming business (the “stick”!) was exploding.

      So how did they — and how can other companies — make such big, bet-the-company kind of moves, without getting mired in fanatical internal debates? Does organizational structure need to change, especially if moving from a more product-, than project-based, approach? What happens to security? And finally, what happens to the role of CIOs; what can/should they do?

      Most interestingly: How will the entire industry be affected as companies not only adopt, but essentially offer, microservices or narrow cloud APIs? How do the trends of microservices, containers, devops, cloud, as-a-service/ on-demand, serverless — all moves towards more and more ephemerality — change the future of computing and even work? Cockcroft (who is now a technology fellow at Battery Ventures) joins this episode of the a16z Podcast, in conversation with Frank Chen and Martin Casado (and Sonal Chokshi) to discuss these shifts and more.

      Show Notes

      • Discussion of how Netflix moved to a microservices architecture [1:26]
      • Security advantages of microservices [8:13], and the general trend toward this architecture in the marketplace [14:21]
      • How development teams and businesses stand to benefit from this shift [18:34]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today’s podcast episode is all about microservices. And I’ve been super eager to focus only on this topic on the podcast, since we mention it a lot in passing, and I’m really excited because we finally get to do that. Our special guest for this topic is Adrian Cockcroft, who helped lead Netflix’s migration to a large-scale, highly available public-cloud architecture a few years ago — making Netflix one of the originators and early adopters of microservices. And Adrian is widely credited for helping pioneer microservices at web-scale. 

      Also joining in the conversation are a16z partners Martin Casado and Frank Chen, who will be moderating the discussion. And in this episode, we cover everything from what [are] microservices, to the evolution of the architecture, to how it changes the shape of organizations, to operations, to changing the role of CIOs. And finally — and this is actually what really excites me the most about this topic — is what new opportunities come up when you have these extremely ephemeral systems that are, you know, just like ghosts in the machine — from containers to servers on-demand, to serverless and what’s happening there, and some really interesting trends on that edge. The conversation begins, however, with the story of how Netflix got into microservices.

      Moving to a microservices architecture

      Frank: Take us back to the days when Netflix had decided they were gonna move to Amazon and commit to a microservices architecture. Let’s pick up the story there. So, what’s it like inside?

      Adrian: We started off basically running away from a monolith. We had over 100 people every two weeks trying to get all the code they’d written in the last two weeks jammed into one codebase, get it through QA, and get that out into production. And that was just getting more and more painful, and we basically decided we had to break it into pieces. You wanted it to be the work of one developer, basically, controlling what they had deployed independently of everybody else. And at the same time, we weren’t looking at moving to cloud.

      Frank: Did you make both big moves at once? In other words, monolith to microservices, and then private data center to Amazon?

      Adrian: Everything together. And sometimes you find incremental change a good theory, but in practice, you have to have a big enough stick to hit everybody with to make everything move at once. And the big stick was, we didn’t have enough data center capacity to support streaming. We were running the DVD business in the data center, on a system that was growing at a respectable rate. But the streaming business was exploding at a much, much higher rate. And because of that, we knew we would have to either build lots of big data centers, or get onto something else. So, the bet was, “Okay, we need to go on cloud. Then what’s the right architecture for doing that? What’s the right organization for doing that?” The developer group is getting bigger and getting less productive, and we wanted to unlock the innovation. So, we were simultaneously trying to get better developer productivity, better time to value — which is one of the key things we’re trying to optimize for, generally. And then there was a whole bunch of other cloud transitions bundled in.

      Frank: As you went from the monolithic application to microservices, what did that entail? What’s that mean? What is a microservices architecture?

      Adrian: Well, originally, I called it fine-grained SOA, service-oriented architecture. And there’s a lot — some people get negative reactions to SOA, because they were out there trying to do it 10, 15 years ago.

      Frank: That’s right. So it’s all the same ideas over and over again with new dressing.

      Adrian: Yeah. It’s a question like, “Why now, and why didn’t it work then?” And if you look at it, what we were doing was — on relatively slow CPUs compared to what we have today, on relatively slow networks, we were processing big fat lumps of XML and parsing it around. And we were really only able to break the application into a few large chunks, because the overhead of all of the message parsing was too high. If you come to today, you know, you can break it into maybe 100th of the size and 100 times as many chunks, because the overhead of the communication is now very low. We’ve got binary protocols. We’re not trying to, sort of, make everything conform to the big SOAP XML messaging schemes. So it became possible to build a fine-grained SOA architecture, and that ended up being called microservices by, I think, Fred George, who was the first to use the word. But it got written up by Martin Fowler, and then everyone said, “Okay, we’ll go with that.”

      Frank: Yeah. So, big-bang moves. This was a bet-the-company set of technology decisions. Looking back at it, what are some of the lessons learned?

      Adrian: I think one of the ways to approach this is to basically create, kind of, a pathfinder or a pioneer team. There was a lot of controversy inside. So half of the company thought this was stupid, and a few of us thought we could make it work and other people a bit more gung-ho. So, we got the people that thought they could make it work into a room and had a one-day project, where we all built a thing in the cloud to see if it would work — built out of the kind of technologies we’d need to use to build this. That team then, sort of, knocked down a bunch of the straw man arguments that everyone else was holding up against us. You know, a lot of the time, it is just straw man arguments, but you have to actually go and build something to actually find out what are the real arguments. Then you discover things you didn’t even know, which are hard. You run into the real blockers, as opposed to the imaginary ones. So, I think the trick is to get a small team, go very deep, discover what you can, and run a whole bunch of these little projects where you’re trying to learn as much as possible with the smallest possible input.

      Frank: You had this cultural aha, which is, “Let’s get the people who are gung-ho about this, and let’s let them go deep, knock down the straw man arguments.” Sort of zoom up to the 30,000-foot view and sort of describe the organization at Netflix before and after. What did it look like before and after, from a skillset point of view, from an organizational design point of view?

      Adrian: This is actually one of the big things that makes a difference. Some organizations are set up already to do microservice-based architectures, and others have to go through a reorg. Netflix emerged naturally out of the way we were structured at the time. We were already structured as small cells that own things, a lot of responsibility. Each team had a very clear idea of what it was building and how it related to the other teams. But it was assembled as a monolith, at the end of the day. So, breaking it apart was a fairly natural thing for us to do. What you see with traditional enterprise siloed organizations is they’re actually having to do a reorg, and set up teams that are responsible for services, and it’s somewhat unnatural for the way they’re currently set up. But I’m seeing an increasing number of people go through that transition. And sometimes you can see it as replacing project-based work with product-based work. So, every team becomes basically a product team for their microservice, and you have the product management aspects and the operational aspects within that team.

      Frank: And did you find that the people who are used to working on the monolith could be retrained, or did you have to have a new crew come in?

      Adrian: The culture at Netflix is interesting. Most of us had been around before. A lot of us had worked on SOA. You know, we’re gray-haired people that had been — there’s a few people that worked at Xerox PARC in the 1980s, and you could go and have arguments within their object-oriented programming. We had some younger people, but it was a lot of very experienced people taking all the stuff they’d learned and synthesizing it together. It was a very collaborative experience. And we came up with things that made sense based on this series of transitions we were going through. The other transition was from a single centralized database. We had this enormous Oracle machine, with a really complicated schema, to a distributed NoSQL database, in the end, based on lots of different Cassandra clusters. And that was the third transition, and that was probably the hardest transition — was getting all of the SQL code and transactional stuff out of the system. It’s actually breaking apart the databases, probably the hardest thing to do — and then splitting chunks of code off is also difficult if you’re trying to pick apart a monolith. And it turns out, if you don’t break apart your database backend, and you just create lots of services that talk to it, you’ve actually created what’s called a distributed monolith, which has all the same fragility of the monolith, and you can’t update things independently, because you’re tied by the database.

      Security advantages

      Frank: You can’t just take the Oracle database and break it up into little pieces. You have to think about it differently. Now, the same thing is true for the rest of the architecture as you migrate to microservices.

      Martin: Yeah. So, I think what excites me about microservices, in general — it moves all of infrastructure up to an application layer. So, if you think about what you normally do in infrastructure, you’ve got these basic abstractions. Like compute and network and storage, which are pretty low level and they’re semantic free, right, you don’t have structured data. One of the huge advantages of going up to a microservice architecture is you can do infrastructure insertion. Things like, for example, security — things for, like, you know, even debugging — basic operations, and management. And you can do it in a way that has the deep context and semantics of the application.

      The point here is that not only are you going away from the monolith, which is really important, and I think it’s great, but also, like, you’ve got more semantics than you’ve ever had before. I mean, this is actually meaningful stuff when you’re dealing with not IP headers, for example, not blocks but actual, like, structured data. And I think that we can actually reimagine a lot of these tools in ways that we’ve never thought of them before, because we’ve never had the ability to have this type of semantics in these toolchains. We’re seeing this burgeoning area of microservices where you almost have, like, a function per company coming up, and now, I believe that all of the old stuff that we had in the internet, whether it’s naming, or service discovery, or routing, or whatever, we’ve got an opportunity to bring this up in, kind of, a much deeper, richer level, which is really cool.

      Frank: Right. So, we were going to the marketplace or the bazaar away from the cathedral, which is, any individual function can be provided by either an internal or external provider. It could be a cloud service. But then, the challenge is, now it’s up to every organization to coordinate, right. And so what are some lessons that you guys have learned along the way of picking best-of-breed and then making sure they work with each other, getting the version control to work?

      Adrian: When you’ve got a monolithic app, everything is in there. If it gets broken into, you have all access. Its connection to the database lets it basically say anything to the database. When you break things into microservices, you’ve got the ability to have some parts of your system be low-security risk and other parts be high-security risk. You can innovate really, really quickly in areas of, sort of, personalization and user experience. And then you maybe have a much more tightly controlled thing for, say, the signup flow and where you’re storing personal information.

      Frank: So, the great news is you have a lot more agility. The price that you pay is you’re doing a lot more coordination. With a monolith, it’s easy. You put all your eggs in one basket, and then, from a security point of view, for instance, you basically just pile a bunch of appliances in front of it. Easy, right? Because it was a monolith. You knew exactly where it was. Now that the perimeter is distributed across many machines, you have to be a lot more mindful of where the attack surface has gone and which security service you need to put in front of that part of the microservices architecture.

      Adrian: So, you cannot have the privilege escalation of “because there is a little bit of PCI compliance needed in one tiny corner of this monolith, the entire monolith is now subject to PCI compliance and SOC 2 compliance,” and all these things. And by splitting it up into pieces, you can have most of your app be extremely agile and very innovative, and then have the bits that need to be safe be extremely safe. And then if you look at the attack surface, you’re basically keeping a very tight control over what can do what. And if you connect them to the databases, you’ve got very single-purpose connections into the database that are doing one thing, and you can start to control at the access level there as well.

      What used to be policy-controlled by the operations people — what they felt was a safe sandbox for the developers — is now really being driven from the other end down. So this idea of developer-driven infrastructure is something that is turning things around. And a lot of what I’m seeing is that big banks, and people like that — they have their existing policy frameworks and rules, and they’re trying to apply it in the new world, and it looks the same, so they’re happy because they’re compliant. But they don’t actually have the real policy separation that they think they have, because it’s all totally reprogrammable, and it’s like you have the illusion that you’re still conforming to the policy.

      A lot of these things were Ops-controlled. So the Ops would control the data center, and then the networks in the data center, and now it’s all developer-defined and software constructs which are controlled by your, you know, cloud APIs. If you’re updating it 10 times a day, there isn’t time to have 10 meetings a day with operations to do the handoff. So, what we’ve been seeing is, people just running it themselves. The only person that knows the exact state of the system is the developer that just updated it. That sounds scary until you realize that each of them is controlling a very small piece of the system, and the aggregate behavior of the system turns out to be really robust and reliable — partly because if you put a developer on call, they write really reliable code, and they don’t release code on Friday afternoons, because they want a quiet weekend. You know, they learn a bunch of practices about what it’s like to be on call and how not to break things.

      Frank: So we went from an in-person change review board, infrequently, right, to vet the changes to continuous change and, “Hey, let’s coordinate over Slack.”

      Adrian: Pretty much. Yeah, you have to tell people what you’re doing, but you don’t have to typically ask for permission and go and have meetings and things like that. This is part of unlocking the innovation. And the people that are most interested in these are large teams of people trying to build complex products, typically enterprises, and they are worried about getting disrupted by the latest Bay Area startup or whatever. There’s an existential threat here, that if you’re doing quarterly releases and your competitor is doing daily releases and continuous delivery, you’re gonna fall so far behind in the user experience that you’re just gonna suffer, right. So, that’s the big driver that is making people say, “Well, how do you get there?” There’s a whole bunch of things tied together. You’re bringing in cloud, DevOps is a whole other area, and microservices as an architecture — all these things tied together — and some cultural change as well in the organization of the company. The companies that are doing well at that are really starting to accelerate off into the distance.

      Market trends toward microservices

      Martin: It’s also worth teasing apart two trends. And one of these trends is, you know, a single company, instead of building a monolithic product, wants to build a microservices product, and gets all the efficiencies of doing that as far as the development process and the OEM process, everything else. But there’s kind of a broader industry trend where companies’ products are basically microservices, right? There’s companies out there that, like, basically, the only way to access the product is through a fairly narrow API. I mean, you know, there’s so many of these now that there are other startups that will just basically stitch them together, and they could build full applications without writing much code. So, I think that, in addition to a single company getting a lot of advantages, I think the entire industry is gonna get a lot of advantages and see a lot of innovation as a result.

      Frank: Yeah. If you had said five years ago that there would be multiple independent public companies that all they do was offer an API, you would have been laughed out of the room, right? And now, look at us. Twilio, and Tribe, and on and on.

      Martin: I like to do the mental exercise of, kind of, where this is all going, and I still love Chris Dixon’s quote of, you know, “Every Unix command becomes a company.” It’s like grep becomes Google, whatever. Like, I think, you know, we may be having an analog here, which is every function becomes a company, right? It’s, like, even more granular than a command line tool. Every single function, or a logical function, becomes an independent company. And I do think there are implications on things like ownership and dependability, and stuff like that, that we haven’t grappled [with] yet as an industry. It’s a very exciting direction.

      Adrian: Yeah. You’re able to build something now that pulls in things from APIs and pulls in some containers, and you just have your little piece of code in the middle that stitches it together and build a completely new service from that. So, it’s just much easier to get things built. It’s more efficient for the big companies, but it has democratized all the way down to pretty much anybody with a laptop can go build something interesting. And if you go back 5 or 10 years, you’re doing things that would be just totally impossible to try and get together at that point. There’s much more room for innovation. 

      It also makes it harder to compete, in some ways, because now it’s hard to build, you know, a billion-dollar software company on top of these things, because they keep changing underneath you, and they’re cheap to build. So, you’ve got lots of disruption coming, and it’s actually, you know, GitHub, and open source is another big player in here that’s just making it much lower cost to get things done. So, what you’re seeing now is Twitter, and Facebook, and Netflix, and Google, and LinkedIn producing the stuff that you actually want to use, which has already been tested at volume, and then it’s actually much harder to build a proprietary software company because you’re competing with these big end users, and you’ve got this thing you’ve just built, and it’s flaky and don’t quite work right.

      Martin: We’ve talked about this, but it seems like closed-source shippable software is on its way out or dead. And there’s a number of reasons for this. One of them is just — the enterprise buyer likes open-source software, but another one is it’s a real burden on the company to ship software, right. I mean, especially if that software is a distributed system, right. I mean, like, you don’t have skilled operators often, every environment is different, right. So, you’ve got these heterogeneous deployment environments. You end up with this, like, the mother of all cache consistency problems, where you’ve got a bunch of versions out there, a lot of products you’ve got to maintain a bunch of versions, etc. It’s hard.

      Frank: The QA matrix from hell, right? Oracle’s version multiplied by the flavors of Unix multiplied by whatever Windows versions you’re supporting, right?

      Martin: Yeah, that’s right, that’s right.

      Frank: Your poor QA manager.

      Martin: Yeah, that’s right. And then distributed systems, generally, I mean, a real trick if you’re running your own operation is you have skilled administrators that know how to manage a cluster. And that, like, there are very, very few companies, and I think maybe one that’s actually managed to ship a distributed system that was manageable with a non-skilled operator. It’s a very, very difficult problem. And so a great thing about — if you offer something as a service is, like, okay, you don’t have any of these problems. And so, like, basically, your post-sales operation budget is way lower. It’s much easier to start a company now, but at the same time, there are questions about, “Okay, so what are the sizes these companies are gonna end up being? I mean, how big is the market for a single function?” I think it’s still to be seen, like, how big these companies are gonna become.

      Frank: Yeah. Big challenge from an investor’s point of view, which is, if the essential argument is, “there will be no more cathedrals, it’s all bazaars from here on out,” it’s a little harder to make money, right, because the biggest…

      Adrian: You’re investing in a food truck, and that’s as big as it’s gonna get.

      Advantages for development teams

      Frank: So, put yourselves in the shoes of the enterprise CIO. The pace of change is accelerating, right. The ink just dried on her team getting VMware certified. And now, we’re on to containers, and then people are talking about serverless and functions as a service with, sort of, Lambda architecture. So, talk a little bit about what’s coming, and then the ability of an average organization to sort of absorb these changes.

      Adrian: Containers came along, really over the last two years, and it’s one of the fastest takeovers of enterprise computing we’ve ever seen. It’s quite remarkable how quickly they were able to colonize the enterprise space. It solved a real problem.

      Frank: What role did containers play in moving away from the monoliths to the microservices architecture?

      Adrian: What happens with the containers — all that stuff is packaged into a bundle which has all the right versions of everything inside it, and you can download it and run it. It also abstracts you away from the particular version of what you’re running on. There’s now containers for Windows as well. But originally, this was a Linux-based concept. You have the same container format if you want to run in-house, or on a public cloud. It doesn’t really matter. That container can run on VMware, or KVM on OpenStack, or on Amazon, or Google, or Azure, or wherever, right. You’ve just abstracted yourself up one level. It gives you that kind of portability. If you think — about machines used to sit at the same IP address for years. People would know a machine — they would actually know the IP address of by heart if they wanted to do something to it, right?

      Martin: Well, I remember that. Yeah.

      Adrian: And then you had VMs came along, and now the VMs are more transient, and you know, this thing would come and go, maybe in, you know, order of weeks or something, a biweekly update of your VM. And then, with containers, it’s perfectly reasonable to have a container that runs for less than a minute. You can create an entire test environment, set it up, run your tests, you know, automatically test it, strip the thing down again, and the size of the things have gotten much smaller. If you just take it to its logical conclusion, we’d basically fire up effectively a container to run a single request and have it sit around for about half a second and then have it go away again. And that’s really the underlying technology behind AWS Lambda. It’s a server on-demand that just isn’t there most of the time. And this is the bleeding edge right now. We have to figure out how to extract these, sort of, ghostly flickering images that are sort of coming into existence for short periods of time. How do you track what’s going on? You end up figuring out how to end-to-end tracing as the only way you can monitor things, rather than being a special case like it is now.

      So, there’s a bunch of interesting problems here, but what’s really been happening is just this trend to more and more ephemerality. And these extremely ephemeral systems — and then the charging. Used to charge by three years’ worth of machine, and then it became, well, you can rent a VM by the hour. And then containers, you know, that’s lighter weight, and now you’re paying by the hundred milliseconds, right? It’s perfectly reasonable to run for half a second, which means that the setup time to create that half-second worth of machine needs to be radically less than half a second. And the time taken to bill for it needs to be less than half a second. If you remember the story of SMS, the SMS record for, you know, 140 characters — the billing record is much bigger than that. It’s more like a kilobyte. So, if you actually take a telco and rip out all of the billing step for their SMS things, you know, it will cost 1/10 of the amount to run if they didn’t bill for it. So, you got this effect that the overhead of doing the thing is actually vastly more than the thing you’re trying to do. So, actually, it’s a really interesting challenge — is how to create monitoring and billing and scheduling systems that work so quickly that you can afford to bill things in timed increments.

      Frank: The portfolio company 21 is, sort of, right in the thick of this, right, which is how do you stand up an ad hoc agreement between an API and an API and, like, have the billing all work. And you know, Bitcoin might play a part in that.

      Martin: Also, to your question, going back to the CIO — I mean, it seems to me, in general, with disruptive technologies, it’s like — the disruption happens first and then all the day-2 Ops happen second. I mean, whatever that is. And, I think, in this case, you know, the disruption is around delaminating the app and breaking it apart. I do think that CIOs should not despair and Ops team should not despair, because what happens very quickly in the vacuum being left from kind of, you know, this sprint on these new technologies is whole, you know, ecosystems and whole industries arise around them to provide visibility, to provide security, to provide Ops, and we’re seeing that now. And so, I mean, I think that it’s quite possible to decouple the disruption — which is this velocity around development — and then, you know, the basic operations. And that tooling is definitely going to happen as well. Understanding that ecosystem, understanding the players is very important if you wanna stay on top of this kind of big change.

      Frank: Leaning forward into the change, assuming the tooling will meet you halfway, right.

      Martin: Exactly right.

      Frank: And then you get the benefit — the big benefit from the CIO’s point of view, in my opinion, is that you don’t have this loop where the business user asks for something. It took you 15 months to build it, only to discover that’s not what the business user really wanted, because the requirements were poorly specified. In these days, right, no problem. I’ve got a change for you, we’ll put it live this afternoon. Right? So, the rapid experimentation that happens in startup plan can now migrate into the big organizations, and you don’t have to get your requirements perfectly specified at the beginning of a waterfall process anymore. Let’s run the experiments.

      Adrian: It’s actually even better than that. What the CIOs are providing now is a set of APIs for the development team that is part of the business to automatically provision whatever they want, with certain policy constraints around it for what they can and can’t do. But fundamentally, you’re providing APIs. Operations has moved from being a ticket driven organization to be an API. They are now no longer a cost center. That is a very profound move, and I’m seeing a lot of these CIOs buying into that. They want to be part of the product. They want to be — how do you support the business? And you provide APIs so that they can just get business done at a rate that you’re not slowing them down.

      Martin: We’re actually seeing the creation of a new buying center in the industry of vertical platform engineering, of vertical DevOps team, whatever. This is, like, budget allocated. It’s actually viewed as a profit center. It’s product aligned, but it’s core infrastructure and operations. And these are very technical buyers, so it’s not the traditional enterprise go-to-market.

      Adrian: This is also moving across industries. We’ve seen, obviously, media and entertainment and, to some extent, retail were early movers — mostly because of the threat of Amazon themselves causing retailers to step up to, sort of, reengineering. We’re now seeing FinTech, you know, or Wall Street is really paying attention. Some people are way down the road, some people are just starting. Manufacturing, that whole industry is just starting to think about this. There’s definitely a, sort of, industry-by-industry, sort of, domino effect as people are figuring this out.

      Frank: So, we’re a decade on or so into this revolution, right. Many strands. What excites you now?

      Martin: For me, what’s really exciting about this, and I’ve said this before — is if we just have the ability to reimagine all of infrastructure, you can now reimagine tooling, and reimagine security, and reimagine operations and management. We get to reimagine it with more semantics and context than we’ve ever had, you know. So, what does it mean to have a firewall in a world where everything is microservices? What does it mean to have operation management, and debugging — things that were traditional boxes, that were stuck on perimeters, now also become functions? And actually managing your infrastructure is almost like looking at a debugger, a context debugger. It’s like you have a symbol table with you. It’s, like, this whole thing is in one large IDE, and you can do that for your operations. I think it’s gonna push the state of the art on how we even think about Ops in entirely new areas. I’m really excited about that change.

      Adrian: I think that the whole serverless area is the bleeding edge right now. The monitoring tools industry is, right now, bring disrupted pretty heavily by serverless. There’s only one or two tools that have really come into existence in the last year or two that have — effectively, a way of processing stuff that is this ephemeral and dynamic. So, there’s some interesting products coming out. It’s just a better way of living. If you’ve a developer and you’re working in the waterfall, siloed organization, it’s kind of soul-destroying for a lot of people, right?

      Frank: Indeed.

      Adrian: And when you get ownership of a product, you know, on distributed teams, you get each distributed team their own product ownership, and they get to define the interface and manage it and run it — yeah, you might be on call, but you’re in much more control of your destiny, and it’s much more rewarding, and it’s more productive. And the ability to get more stuff done as a developer is just rewarding anyway, right. It’s a better way of working for people.

      Frank: Well, that’s great. Well, thank you, Adrian. Thank you, Martin. We’ll see a lot more unfold as the architecture shifts.

      • Adrian Cockcroft

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      Pricing, Pricing, Pricing

      Mark Cranney, Martin Casado, and Scott Kupor

      “Raise prices.” Regular listeners of our podcast have heard this advice more than once. But why is this so key and yet so hard for many technical founders? And how should startups go about raising prices — or more specifically, creating value — for their products?

      In this episode of the a16z Podcast, former sales VP Mark Cranney (and head of a16z’s EBC and go-to-market practice for startups) and former startup founder (and general partner focused on all things infrastructure) Martin Casado talk to managing partner Scott Kupor about pricing for startups … especially for category-creating businesses. It’s not all “pricing, pricing, pricing” though — there’s another important “p” in there too!

      Show Notes

      • Why it’s important to price aggressively from the outset [0:43]
      • Using salespeople to gather information about the marketplace [10:07], and how to choose the right salespeople [16:50]
      • Discussion around packaging products [20:57] and how to handle pricing errors [25:51]
      • Scheduling pricing reviews and other sales strategies [29:16]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Regular listeners of the podcast have probably heard us say more than once that entrepreneurs should raise prices. Why? And the other question that comes up often is, how? On this episode of the “a16z Podcast,” we talk all about pricing, packaging, and more. Joining the conversation, our general partner, Martín Casado, who was formerly the co-founder and CTO of Nicira, which was later acquired by VMware. And most recently, he served as a general manager for one of their major business units. Also joining is Mark Cranney, who heads up our go-to-market practice and our EBC, or our Executive Briefing Center. And he was a former VP of sales. And moderating the podcast is managing partner, Scott Cooper.

      Price aggressively

      Scott: Hello, everybody. This is Scott Cooper. I’m here with Martin and Mark. And we are here to talk about pricing.

      Martin: One of the reasons that this podcast, in general, is so important is, I think that pricing is one of the least intuitive things for technical founders, having been one. And it certainly was my bias, you know, coming out of, like, my Ph.D and doing a technical startup, that you almost always want to, like, take whatever your technology is and get into everybody’s hands — and then you kind of assume later on that, like, you can somehow monetize it. Ben Horowitz was on my board at the time. And I was explaining, I said, “Well, I think that we should kind of enter at a very low price, and that way we’ll have more people that use it.” And he looks at me with his very stern look and he says, “I want you to be very careful, because no single decision will impact the valuation of your company more than the decision you’re about to make on pricing.” And so, that kind of started my foray into pricing.

      Scott: As long as you’ve got the mic, I mean, let’s start there. So, what’s wrong with that? So, why not be cheap? And then, why can’t you just raise prices later? What’s the problem?

      Martin: So, there’s an interesting thing, especially if you’re dealing with pre-chasm-type markets. So, pre-chasm means you’re bringing a product to market where there isn’t a market yet, or the market is very immature, right? So, there may not be a budget. There may not be a buyer. They may not know how to think about it. And so, here’s the fallacy that many technical founders have, and I had as well. I think if you build technology, there’s intrinsic value to it. This is worth so much money because it’s intrinsically valuable. And that’s not generally what happens. What happens, just in human psychology, is actually, like — [people] will set the value on whatever they’re getting based on how they acquired it. And so, in early markets, nobody really knows how to value what you have. And so, it’s very important for you to establish the value in the market. Otherwise, you end up devaluing yourself right away, and you’ve just cannibalized your top-line revenue almost immediately.

      Mark: The scenario you described to begin with is extremely common. And the reason it’s common, over and above what you’ve outlined is, in a lot of cases, that technical founder has not put themselves in the buyer’s shoes to understand what that prospective buyer needs to go through. Because like you described, they don’t know what the criteria is, and how they should look and/or evaluate your solution. They might not even know it’s a solution, because they don’t know there’s a problem. That’s kind of the job of sales and marketing — is to go put yourself in that buyer’s shoes, understand what their as-is environment is. 

      Answer those questions along that sales process of, “Why should I even do anything, to even evaluate you or look at you?” And then “why you” and then “why now?” The “why now” piece is the piece where the pricing really starts to come into play, and that’s where you’ve actually gotten enough information out to understand what their return would be — what an ROI would be. And if I can’t go show what that value is, and not have that deep information about what that prospect’s costs are gonna be or what my impact of their business is gonna be, then, yeah, it’s easy to say, “I’ll give it away for free and we’ll monetize later.” So, that’s where that comes from, that natural inclination to, “I’ll figure this out later.”

      Scott: Mark, so, you’re kind of describing, kind of, value-based pricing, right? So, does that mean in those early days, then, that you kind of iterate on the pricing with the customer, or are you suggesting that people do that analysis and then present that to customers, or is it really just iterative.

      Mark: Well, I mean, a lot of it depends on what exactly your — you know, what is your value proposition. If there’s some kind of standards that are out there, and the customer is used to acquiring a solution, you know, and your solution is gonna be similar, there’s models you can bounce off as far as what’s going on in the as-is world. What Martin is talking about — what he was doing was so transformational, it was hard for a customer or a prospect to get their arms around. And Martin maybe didn’t have enough of, you know, the “as-is” — just intuitively, having been in these larger environments, to really, you know, early on understand what that would mean. And there’s a long development cycle before that value would even be delivered. Right? So, the value-based pricing, I think, is for sure true. But part of it’s just — you gotta go through a whole sales process before you can start to get comfortable and have pricing control.

      Fast forward a little bit, you know, where you’ve gotten or you’re down the path of getting product-market fit, or you’re beyond that — you know, I see a lot of entrepreneurs that kind of get stuck in the mud on their pricing, or they’ve gone in too low, they’re not sure how to go start to raise pricing. A lot of that is more in the packaging. What is your roadmap? What sets of functionality has the prospect or the customer already realized value from? And then where are you going with the product? Is that something you wanna just charge the same price for, or do you need to start chunking that up, because they might be buying different ways?

      The customer or the prospects asking for additional, you know, features and functionality, or scalability, or architecture <inaudible>, that shouldn’t be the same price. So, I need a different type of packaging and a different pricing model to go build on that. And then there’s the option of, like, “I wanna start a la carte,” versus, you know, in a package all at once. Some customers might not need the whole product when you’re first starting out, or they don’t need it in a user workgroup setting. But the enterprise, as they adopt, will. So, you’re holding some functionality back and waiting for them to want that, and then go establish that value to do it.

      But I totally understand where Martin is coming from early on, because I see it over and over and over again. And it is, in a lot of cases, the value of the salesforce. The other thing I sensed in his initial answer was, you know — I don’t wanna go, you know, spend the time or the money. And a lot of that is putting the boots on the ground or, you know, inside in the marketing to go, you know, pay for that. But you’re paying for it either way. Right? You’re paying for it by giving everything away for free, because you’re not investing in sales and marketing — or you’re gonna have a channel partner do it, which you’re giving your margin away. Somebody is paying for it. And if it’s something that’s just so new and differentiated and not defined, and it’s gonna require, you know, the customer or somebody’s gotta go in there and kind of really tease out, “How are you doing things now? What is the pain?” They don’t know there’s a better way of doing it. That’s kind of our job as entrepreneurs, and/or as a sales and marketing organization, to be the translator between, you know, kind of the old world and the new world. And that’s where you start to understand, you know, how to piece this thing together from a value-based standpoint.

      Martin: I think you can roughly, like, dissect the world into two pieces. You’ve got, you know, market category creation. Your constituency, whatever they are — they wake up in the morning and think about everything, but not your thing. Right? It’s not even something that, like, they consider. And there’s mature markets where you’re entering an existing market where pricing has already been set through, you know, a lot of transactions that have happened. And so, I think in the, kind of, post-chasm or mature market world, I think that there is existing pricing. There are comparables that you can work. But in the pre-chasm world, the thing doesn’t even exist. So, not only you’re describing that your thing exists, but you’re actually trying to attach a value to it. I’ve gotten so many times the question — it’s like, “Okay. So, how do you set that initial price?” You get a lot of these PMM types, they wanna go do this market research, and, like, all the stuff. But it’s very difficult to do research on something that doesn’t exist.

      I mean, if you think about a lot of pre-chasm work from the technical side, you’re really being prescriptive, like, you’re not really asking the customer what they want. So, here’s my experience, you know, over a couple of products. The only way I’ve been able to establish pricing in a pre-chasm market is, you start pretty high and then you let the salespeople shake it out. You’ve got really, really good salespeople that go in there, have the dialogue, have the discussion, understand it. And it’s this really iterative process, where you’ve got the sales guys piped into the nervous system of the product development side, and then you get a sense for, kind of, what the market will bear. I don’t think you can have, you know, a bunch of MBAs out there doing research, because this is so new. And so, I don’t know, Mark, like, this is something you’ve done a lot of. I’d love to know your thoughts on that?

      Mark: Well, I definitely agree with the start high. It’s way easier to go down than it is go up. When you talk about pre-chasm, the first thing you wanna do is really segment and target and get to those point-of-the-spear-type prospects. They’re gonna be quicker to understand that there’s probably a better way, and maybe have gone down that build-it-yourself-type path where they’ve got the recognition that there is another way. That’s where, you know, maybe kind of one of the first places is to start, from a segmentation and targeting. 

      They’re gonna be those early type customers, as you’re going through that product-market fit, where you need to be able to understand what it could mean financially, and to also partner with them on what that pricing could be — because they’re the ones where you’re gonna get the most information and input, as far as what the value is gonna be. And it’s always gonna be involving — from a pricing standpoint, in some cases, we may be lowering the price of, like, the foundational piece of our technology, because the market is changing, right? Things are gonna get more commoditized, but we’re racing upstack and charging for new things that are more valuable. I mean, if you think about the whole lifecycle of pricing, you know, in a lot of tech companies, it’s not just where it starts, you know, it’s something you’re constantly — should be looking at.

      Gathering intel from salespeople

      Scott: So, just to put some meat on the bones, then. So we’re saying, “Look, start high,” you know, to Mark’s point, which is, “Look, it’s always easy to go down.” It is an iterative process. But within that context, there is some framework for how to evaluate the value-based pricing, right? So, in the Nicira case, presumably there are things that engineers can now do in terms of changes to the network that they couldn’t have done before. And that has demonstrable business value, or there’s either fewer resources that are required. There’s some kind of framework where we can begin to work with the customer to say, “Hey, the value that you would get in the organization, either in headcount cost reduction, or flexibility, or new product rollouts or other things, equates to some portion of that being captured through the software.”

      Mark: Well, one thing, maybe there, just to keep in mind is, from a pricing standpoint, you know, if you think about a large enterprise — if you break the audiences down, you know, the pricing to, like, a user, like, say in Martin’s case, you know, like, the early people — he was probably talking to network engineers. That kind of intel is gonna be completely different than if you’re up at the top of an organization with the CXO that has a wider view and, you know, is gonna understand a bigger story. And they’re also gonna understand, you know, when you get into headcount reduction, or you get into, you know, different types of ROI-type modeling, you know, what you’re getting credit for around productivity from a financial standpoint or a pricing or a value standpoint is gonna be completely different with mid-level managers across multiple functions, all the way up to a CXO with higher-level initiatives. So, that’s something that maybe a first-time founder, that hasn’t had to go through that whole process — it’s not gonna be intuitive right off the bat.

      Martin: Just to add to that. I think it’s just really seductive to think that like, “Oh, I’m an analytical person, and so I can do some basic research. And based on my research sitting in my, you know, offices in San Francisco, I know how to set the pricing, because I’ve got comparables and yada, yada, yada.”

      Mark: I have my buddies, too, that’ll tell me.

      Martin: That’s right. I could talk to my friends. They’ll tell me, like…

      Mark: Not a bubble. I mean, it’s not a little bubble out here. I mean, everybody is gonna…

      Martin: Everybody.

      Mark: …like this, right?

      Martin: And that’s actually kind of where I was initially, seriously. I am such a convert for a couple of reasons. One of them is, like, the value of a sales team is certainly to sell and bring in a number. But in my experience, when it comes to setting pricing and understanding what the market will bear, like, there’s nothing that can do it except for sales. This is just my experience. It’s not market research. It’s not marketing. It’s not the entrepreneur. I don’t believe, if you’re doing real category creation, you can just build an ROI tool. I don’t think it’s that simple. Depending on who you talk to in the organization, you’re gonna get, like, two different understandings of what the value actually is.

      Mark: Some people aren’t gonna wanna be held to that ROI, right? They’re taking a risk. As you go up in the organization, they’re gonna want a bigger potential return for making this type of a bet, and the risk profiles are gonna be completely different as well.

      Martin: Yeah. And I think something that entrepreneurs underestimate is large companies’ appetite to learn from startups. Very often in a hot area, say, AI or deep learning, you’ll have these, like, really smart entrepreneurs that have done their Ph.D, maybe they just peeled out of Google or something. Companies will pay to engage with them. They may pay 100k or 200k for a POC. And so, the entrepreneur is like, “I’ve got product-market fit. These guys are talking to me. The pricing is set.” But the reality is that it makes total sense for the company to do this and to learn from them. And so, I mean, one thing I really learned to appreciate about, you know, setting pricing aggressively early on, is you start to get real market feedback. You can’t delude yourself anymore. People aren’t buying it to learn about it, they aren’t doing this because you’re super charismatic. You get real signals. And the reality is early on in a company’s lifecycle, you really can only take on so many customers anyways. Setting pricing high really helps that.

      Mark: Fast forward a little bit. Let’s assume we’re getting past POCs and our first 5, 10, 20 customers, but, you know, I see a lot of situations that we’re getting stuck in the mud with a lot of companies. And the reason they get stuck in the mud, particularly dealing with these bigger companies is — a lot of earlier stage companies, they haven’t thought through what the rest of that deal is gonna look like if it expanded throughout the entire environment. I see it time and time again and I constantly press. 

      I say, “Look, in almost any proposal-type situation, you want it to have not only your initial pricing per unit set, but you need to go model and understand what would happen if that customer adopted throughout their entire enterprise.” Now, early on, it sounds like a pipe dream and, “Oh, my gosh, I can’t think that far ahead.” But you’ve got to go, literally — kind of, model these things up, because that’s what the customer is gonna be asking themselves. “All right. If I do this proof of concept for 100 nodes or 100 users, but I have 100,000 in my environment nodes or users or whatever the model is,” they’re gonna be thinking ahead, because they’ve been through this game before.

      So, when you’re starting that quote process, to take a lot of friction out of the whole buying and selling process — to be able to, kind of, give them, you know, in the quotes, “Here’s what the small deal would look like to get started <inaudible>. Here’s what a medium-sized deal [would look like] with multi-BU. And then here’s what a large deal would look like.” I’ve technically validated it. I know I can scale. I know I’ve got the architecture or whatever. And I know I’ve got the security and things. But I also, I can actually back that up from a — you know, business case standpoint.

      Martin: You’ve talking about pricing, which is great, but there’s organizational issues. Obviously, they’re incredibly important in selling.

      Mark: There are different models. Right? I’m replacing something that’s been on-prem — now we’re offering SaaS. Two different accounting — I mean, we’re accounting for things different. In the old way, I capitalize it. In the new way, it’s coming out of operating. And so, that slows people down in a lot of cases, just from a financial chops standpoint, and dealing with the customer — and it changes the budget. The other thing that’s different is, I mean, that might have been centralized before with, you know, IT, but the whole world’s changing because it’s a line-of-business-type sell. People get stuck in the mud all the time,

      Martin: I think it’s really worth underscoring the point Mark is making, which is, like, you work so hard in a startup, like, moving the ball an inch that often you don’t really consider, like, the macro success scenario. Because I would go in saying, “Oh, boy, wouldn’t it be great to get to the POC? Wouldn’t it be great to get to the initial sale?” But in reality, they’re trying to evaluate the full-on risk, which assumes that they like the technology and they’re gonna adopt it, which means they’re thinking all the way through it. And so, if you don’t walk in having thought through, “What would it mean in the full success scenario?” it’s much more difficult to have the conversation. And this is a trap, I think, entrepreneurs fall in all the time, not just in sales, but generally just trying to be incremental in the way that they build things.

      Mark: That kind of goes into — it’s a competence and a confidence-type situation as well. And I might have technical competence around, you know, whatever my solution is, but I don’t have that customer knowledge competence. In a lot of cases, we’re trying to just throw a number out there, because we don’t have any data for that number. We don’t have any confidence in what it’s costing them, or what it would mean to them from a transformative standpoint. So we’re kind of guessing. And some of that might just be a lack of knowledge, and/or lack of willingness to go invest in, you know, the people and the processes that would, you know, be able to bring that to you. But you see it even with companies that do have sales and marketing organizations. And so, I’d be a little careful to say, “Hey, any sales and/or marketer is gonna be able to figure this thing out for me?” Because it does take somebody that’s a little more intuitive that it isn’t taking shortcuts, you know, somebody that wants to go understand and create the value.

      Martin: I’ve actually found roughly two types of salespeople that roughly align with the maturity of the market. In a mature market, the customer is already educated. So, like, the type of salesperson that excels in that environment is very different than in the early days. I actually think that the actual competency of the salesperson depends on the size of the market. I mean, different salespeople are optimized for different types of markets.

      Mark: I totally agree. One trap a lot of CEOs or founders might fall into here is, “I’m gonna go hire somebody that’s selling something similar to what I sell.” And the problem with that, like in your case — if I wanna go, you know, just hiring all network guys. Remember, I probably screamed at you and…

      Martin: Oh, you did. I remember…

      Mark: …the CEO back then…

      Martin: I remember the conversation.

      Mark: …who was — sometimes they are already native. And what you are doing and, in a lot of cases, what other technical founders are doing is so transformational that you’re not even gonna be able to sell that salesperson and/or those teams in a lot of cases, because they’re not even gonna believe it themselves.

      Martin: Yeah. But not only that. I mean, this is something that you told me, which is, “Listen, you can’t take someone that’s selling into a mature market and put them in an immature market.” Here’s the big trap that — and I totally fell into this, and I think a lot of first-time founders fall into is — often the mature market salesperson is really compelling.

      Mark: Yeah. And I probably might have gotten a little — well, I apologize. Five, six years later, I’ll apologize now.

      Scott: Because you’ve changed so much between.

      Mark: I’ve changed so much. Actually, I’ve gotten worse. The problem — you’ve really gotta go find somebody that really enjoys and has that intellectual curiosity, and that deep understanding of the customer and/or will tease that out. Somebody that gets excited about doing this exact thing — it’s typically not the one that’s been running in somebody’s playbook. It’s the one that really understands how to go create that playbook from scratch, and somebody that gets excited about this. With the founder and the technical vision, they can go map up to that, and translate that to, “Wow, that could be really transformational.” That person that can take you to those early potential targets and prospects, turn them into customers, learn, stop, put the recipe — you know, take the recipe, make it repeatable, turn that into a playbook, make that scalable. You can’t move it out of the lab, you know, into production unless you’re…

      Martin: It’s infrastructure, of course. Yeah.

      Mark: …you’re pretty sure, right? That’s a whole different profile than, you know, grabbing someone off the shelf that, you know, has been on a route-sell for their whole career.

      Martin: So much of the early customer engagement should be, like, figuring out the product-market fit. So, you really want market feedback that you can use. And if you have somebody that, like, you know, every meeting is a good meeting and they’re using relationships, this and that, I don’t think you get real feedback from the market. But if you got, like, a good hunter that’s looking for the real opportunity and the large deal and will fight for it, I think the business gets that feedback. So, I think it’s so critical to hire the right type of salesperson early on.

      Packaging products effectively

      Scott: So, talk more about packaging. How do people think about new functionality? How do they think about upgrades to the existing functionality? How does all that play into it?

      Mark: It depends on what kind of product we’re talking about. But is that customer early on gonna need everything that’s in the product the way it’s packaged? Or, in some cases — let’s say there’s three buckets of functionality, but in a lot of cases, depending on the user, maybe some buyers only need one. But as it expands, they’re gonna need two, and/or maybe eventually they’ll need all three. So, should we chunk that up to make it easier for them to get started, even though they don’t need the other functionality, or they don’t recognize that need yet? So, there’s — maybe we should have three prices for an a la carte type version, and then maybe we have a package if they get it all up front. But then the other thing is, let’s take the product roadmap and let’s look ahead. Let’s take the feedback we’re getting from the market, understanding what’s coming down the pike, or what we’re thinking about building.

      And I think this needs to be a closed-loop process from the field, with product development. As you mature, you have product management, product marketing. All these groups need to be working together and, kind of, challenging each other as, “Is this something that should be added? Is this something the customer is gonna get value out of that we should be charging for?” Because we can put an ROI to it, and we can make it easier to buy early on, and they can grow into it later on with the packaging and/or different options. The other thing is that, you know, one solution in one vertical and/or use case, from a pricing standpoint or a value standpoint can be completely different in another. So, you’ve gotta kinda balance those types of things from a value-based standpoint. You’ve gotta have the knobs just ready to crank. I think a lot of companies wait too long and/or they drop the ball on it.

      Martin: So, you’re doing your company, you’re creating your product, you know, you’ve set your pricing pretty aggressively, you’ve established a price in the market, you’re very happy, but you’ve probably necessarily priced yourself out of some constituencies, right? So, now you wanna go ahead and expand your footprint. You maybe want to go to, like, other areas or other verticals. And you wanna do this in a way where you maybe have tiered pricing, but the risk of tiered pricing is cannibalization. If you don’t know you’re gonna do this beforehand, you may not have the flexibility to actually pull out independent bits of value. And so, now you’ve got — on one hand, either you cannibalize yourself with, like, an over-feature-rich product for the lower price, or you don’t get sufficient market expansion because you’ve priced yourself out of it. And so, like, I think that — absolutely right. Early on, you should make sure that you’re thinking through how you can bifurcate this when you do your market expansion.

      Mark: Yeah. Another good example on that is, you know, those early customers sometimes if you’ve really — if you’ve done a good job, you’ve got those early wins, and you gave them a sweetheart, lighthouse-type deal, you’ve really gotta pay attention to not just the pricing but how you’ve gone and contracted. Because later on, I’ve seen it happen over and over again, companies just kicking themselves in the tail because they really gave up the farm.

      Scott: Because some of this, you’re right, is a product packaging issue. Have you literally sold everything that you will ever build in perpetuity to this customer upfront?

      Mark: Right. Let’s go get those lighthouse customers, but let’s put a fence around the “the enterprise” thing and — a lot of cases, it’s just the startup is marching up to this big bad wolf, and come on in and, you know, I’ll huff and puff and blow your house down. And guess what? Yeah. These big companies know how to negotiate and contract, and they know every trick in the book. And we’ve seen some of these teams walk in with their junior varsity uniforms on, and then there’s a lot of injuries that happen. So, you really gotta be careful. So, how do you solve that? I mean, get some help and make sure you get the right advisors helping you with these early-type deals.

      Martin: Yeah. I think it’s important people understand how deceptive this all is, because many large companies have outreach programs to startups. So, if you engage with a finance company, like a large bank, they’ll be like, “Listen, we work with startups all the time.” They have whole groups that will take these things and POC them. That, in my experience, never actually makes its way over to the procurement office. So, while they’re great at, you know, engaging with you, lightweight process, POC’ing, finding value, you meet the technical teams — you even have a deployment scoped out, and a professional procurement person in the room. And if you don’t know what you’re doing — I mean, I’ve been in multiple situations, we were very close to basically getting site licenses to, like, 100,000-person organizations for almost no pricing. And this is when, again, Mark comes in and kind of sets us straight. But it’s very, very important to be sure you know for every one of these large deals what you’re doing.

      Mm: It’s pretty out there in the forest, but you will get eaten if you’re there after dark.

      Correcting mistakes

      Scott: This is all good advice. How do you fix problems or fix mistakes? What if we’ve gone out and we have priced too low, or what if we’ve, you know, discovered that we essentially haven’t ringfenced people and we’ve given them stuff? How do you approach that problem? How do you think about it? Are there things, at least, you know, in retrospect, companies can do to kind of right the ship in those scenarios?

      Mark: Yeah. I mean, it’s really — I mean, it’s case-specific. A lot of it is in the language of the agreement that you’ve done. I mean, if you’re doing term licensing or subscription-type deals, you know, that’s different than a perpetual. That’s another piece on this contractual thing. The buyers are saying, “We’re gonna do it on our paper” and that type of thing. The startup doesn’t have any paper. They don’t even — I get these questions all the time. It’s like, “We’re trying to figure out how to deal with these guys.” And they want us to mark up their SLAs and their MLAs and, you know, master license agreements and stuff like that. And we don’t really know what to do. And so the reason you don’t know what to do is you haven’t put the work in to kind of figure out what yours is gonna be. So, the ideal situation, as quick as you can, is you gotta think this stuff through, and you need to have your language. And look, early on, particularly even later on when you’re a big company, you’re gonna swap paper and everybody’s gonna mark it up. But you should know what you need in, from a language standpoint, to protect yourself. As far as redoing the deal, it’s just customer or it’s case-specific.

      One thing I’d like to add, though, along those lines, though. On the product development side, if you think about this early enough — and it doesn’t have to be right at the onset, but as you grow the product and functionality out — from a product standpoint or there’s things that can kind of turn on and off to help me modularize. You wanna think about how I can do that, particularly — and you get the bottoms-up models. There’s a lot of nuance in that, as well, and there’s always the resistance and/or in the DevOps-type environment on the open-source, as you’re moving up the stack to the premium and/or the enterprise-type pricing and functionality and support, that if you’re not constantly understanding where you’re at with the customers and/or in the competition a lot of cases, you’re probably put yourself at risk.

      So, having, on the product side — be able to turn things on and off or be able to measure and monitor and be able to not only establish but measure that value over time. I see a lot of companies, even big mature companies — they’ve done all the front work. They go get the deal. Everybody agreed on, you know, what the value was gonna be. One of the most valuable things is to go back and validate that, and put that into a case study. And the best place to sell something is where you’ve already sold something. Right? So, they’re constantly working with the customer and with the engineering and product development, to kind of bring the best value to the customer — and then measure it and make sure everybody is on the same page. And then, you know, that helps the next 10, 20, 100 customers come on board too from a sureness standpoint.

      Martin: Scott, to your question about, like, setting pricing in the market. I mean, the bad news about setting pricing in a market is, it’s really hard and it takes a long time to actually set the price in the market. The good news is, it takes a long time. So, if you enter at too low of a price point, like, it takes a long time for these things to solidify. And so, I do think that there are options to kind of raise price, you know, add differentiation based on functionality — especially early on in the product cycle. So, it’s not, kind of, like a misstep that you wanna do, but if you do it, I do think there’s a lot of time to correct it.

      Mark: I agree. I totally agree. There’s gonna be a lot of trial and error in that process.

      Pricing reviews and sales strategies

      Scott: Yeah. I wanna touch on two other things quickly first. One is just — so we’ve been talking about, kind of, initial pricing and packaging and stuff like that. How do you think about, kind of, the ongoing process? So, how often should CEOs and the VP of sales and VP of product management be thinking about, “When do I revisit pricing?” Is it on a release basis? Is it new competitors enter the market? What are the things that people should be thinking about that give them some guidance as to, you know, how often or how frequently you think about these types of things?

      Mark: I think it’s almost constant. Again, a lot of it depends on the stage. But if you’re not reviewing, you know, a win-loss type report on a monthly, at least a quarterly, you know — some kind of fairly frequent cadence depending on what’s going on in your business and in the market — and doing the post-mortems on both sides of that — I think you’re putting yourself at risk. And mapping it up with, “Where am I at with the product?” or, “Where am I at with my customers?” That whole roadmap discussion isn’t just an internal discussion. That’s something that you’re constantly doing with your early customers that may be partway down the journey to fully deploying your solution. 

      You’re able to check that from a pricing and functionality standpoint. Then, what [does] the competitive dynamic and landscape look like? I might be getting, you know, pressure from below me — maybe it’s from the open-source world, or the homegrown, maybe it’s from lower-end competitors that are just doing small medium, and you’re starting top-down in the market, you know, with the bigger companies. It may be from incumbents that are reacting and starting to, you know, understand that you’ve done some damage to them, and they might be adjusting what they’re doing. So, a constant review, I think, of that is — and, really, a 360-type situation — is something that should be pretty frequent.

      Scott: Another question I had was, you guys have also hinted this a couple of times, but the relation between the type of sales organization you can support and what kind of pricing you have out in the market. And I don’t know if there’s any heuristics we can give folks, but, you know, in order to be able to support, for example, a direct enterprise selling effort — which requires, you know, a lot of, you know, high touch — there are probably certain ways you have to think about pricing and what your average selling price is, and what you can get an account, versus obviously something that might be done through an inside sales organization. So, is that something that, kind of, CEOs and, you know, VPs of sales and VPs of products need to think about from the very beginning? Which is, how does pricing relate to the actual mechanism by which I’m gonna go to market? And how should people think about that?

      Mark: Yeah. I think it’s pricing, it’s packaging, it’s my product-type strategy. If I’m gonna be a bottoms-up, I mean, it’s gonna start with, you know, some marketing and, you know, a freemium to premium type model, and I’m gonna work on that adoption, and then convert them from freemium, more to the premium. And later on, it’ll be enterprise, then that’s — because my transaction and my ASPs are gonna be extremely small, then that’s gonna drive your go-to-market. A lot of it is product-related. If it’s something that requires you to come in high and move left and right, and be agile in organization, as well as up and down — and because of the complexity is — that typically means there’s also gonna be a big outlay, either kept capital end or operating, and that’s probably gonna require, you know, an outside direct salesforce. The pricing better — and the size of the deals are gonna have to map up to support that to go fund that.

      So, you can’t have a mismatch. In some cases, you’re gonna kind of have [it] all, right? You’re gonna have, you know, the ability to sell across all. And you might have to put in the layers of your go-to-market, you know, from a sales and a marketing standpoint to build that bottom-up. You know, maybe a mid-market-type or commercial-type teams and more of the enterprise major account type named account situations, and different types of marketing to drive different types of penetration across the board. So, it really hinges around the product. And it also can hinge around what your product development strategy is, and what a competitive landscape might look like. I mean, I can think of tons of examples.

      Martin: Yeah. Often in technical organizations, sales basically dominates, like, costs because it’s a variable cost. We need more sales to bring in more money type thing, where R&D is often more fixed. And the cost of, like, an ISR is gonna be much less than a direct sales force, but only certain types of products or markets are amenable to an inside sales model, right? If it’s a very mature market and the customer is educated, it’s probably more amenable than if it’s something totally new. Also, if it doesn’t require a lot of integration, or isn’t super technical, or the product is made very simple to use, it’s more amenable to an ISR. An ISR is an inside sales rep, which is basically someone on the phone which will call, rather than, you know, has a briefcase…

      Mark: Lower cost.

      Martin: …hops in an airplane. It’s lower cost.

      Mark: Inside. Yeah.

      Martin: What we’ve seen in our portfolio companies — a number of them — is actually they’ll experiment with both. So, they’ll start with ISR and a couple of fields and they’ll actually play with the model to understand what works best, especially if they’re moving towards more non-traditional buyers for IT. And so, we’ve seen this in a number of companies. And they’re able to determine over time which model is the most cost-effective.

      Mark: Yeah. I also see, in a lot of cases, you know, it starts one way and they wait too long to add the other layers in. I mean, it’s very common on the inside-type bottoms-up motion that, particularly as they build out the management, that sometimes they’ll miss moving up market because the VP might not have that skill set, you know, to go build out that next level. So, that’s something I think the first-time CEOs need to understand and be questioning themselves, and get help to question. “Am I slowing the whole company down because I’ve, you know, under hired?” Again, it depends on the stage of the company, but you should look at and calibrate, and look at both, you know, “What does an under versus an over-provision look like? And how much headroom am I gonna have?” because you could damage yourself by waiting too long.

      Scott: So, we’re talking a lot about, kind of, first-time CEOs here. And let’s just assume the product is not taking for some reason, it’s not selling. Yeah, we’re missing our plan. As a CEO, how do you know whether you have a product problem, whether you have a problem with your sales team, whether you have a pricing problem, a packaging problem? How do you tease these things apart in a way that actually helps you think about ways to address issues like that?

      Martin: The one bit of insider advice that I would give to first-time CEOs or entrepreneurs is, it’s really hard to find product-market fit. And it’s a saga that can last for years and, you know, you’re gonna doubt yourself, and you’re gonna doubt the product, and you’re going to doubt the market, and all sorts of different feedback. And especially if you’re doing, like, serious category creation, I mean, it takes a long time for markets to mature. Markets mature at their own pace. So, I don’t have a simple answer for what to look for, but I do know that you have to be patient and you have to be persistent and, you know, it takes a while. And I also know, having seen it, once you hit the inflection, it’s really obvious. Once you hit product-market fit, you start to get more engagements than the organization can handle, and you can’t scale enough.

      Scott: Mark, Martin, thank you for the time. It’s time for us to wrap. So, just to kind of encapsulate what we talked about, you know — price early, price often, this is definitely an integral process. Get your sales guys in and the right sales guys, right, whether they’re the kind of hunter or gatherer folks in early to help you with this process. And then, you know, kind of, the important point that, you know, Mark always makes, which is, you gotta think about packaging, right? Don’t give away, basically, you know, the entire collection of stuff that you will build for the next 20 years to your first customer. So, think about how to segment it both in terms of users as well as features and functionality. So, lots of things for people to chew on here.

      Martin: And be aggressive with pricing.

      Scott: Be aggressive with pricing. That’s right. Thank you.

      Mark: Thank you.

      Martin: Thank you.

      • Mark Cranney is the COO of Skydio. Previously he served as COO at SignalFx, and prior to that was an operating partner at a16z where he oversaw the market development team.

      • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

      • Scott Kupor is an Investing Partner at Andreessen Horowitz where he is also responsible for all operational aspects of running the firm.

      The Meaning of Emoji

      Fred Benenson, Jennifer 8. Lee, and Sonal Chokshi

      This podcast is all about emoji. But it’s really about how innovation really comes about — through the tension between standards vs. proprietary moves; the politics of time and place; and the economics of creativity, from making to funding … Beginning with a project on Kickstarter to crowd-translate Moby Dick entirely into emoji to getting dumplings into emoji form and ending with the Library of Congress and an “emoji-con”. So joining us for this conversation are former VP of Data at Kickstarter Fred Benenson (and the ???? behind ‘Emoji Dick’) and former New York Times reporter and current Unicode emoji subcommittee member Jennifer 8. Lee (one of the ???? behind the dumpling emoji). 

      So yes, this podcast is all about emoji. But it’s also about where emoji fits in the taxonomy of social communication — from emoticons to stickers — and why this matters, from making emotions machine-readable to being able to add “limbic” visual expression to our world of text. If emoji is a (very limited) language, what tradeoffs do we make for fewer degrees of freedom and greater ambiguity? How exactly does one then translate emoji (let alone translate something into emoji)? How do emoji work, both technically underneath the hood and in the (committee meeting) room where it happens? And finally, what happens as emoji becomes a means of personalized expression?

      This a16z Podcast is all about emoji. We only wish it could be in emoji!

      Show Notes

      • How emoji originated, how they’re standardized, and more [0:48]
      • The difference between emoji and emoticons [11:03]
      • Social and political considerations around which emoji to include [15:30]
      • Using stickers and images to express emotion [21:40], as well as Bitmoji [24:20]
      • The story behind creating “Emoji Dick,” a translation of “Moby Dick” using emoji [29:11]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. Today’s episode is all about emoji. But it’s also about bigger questions and how innovations come about, from the tension between open standards and proprietary systems, to the economics of creativity. We begin with a tour of different emoji and how they came about, the politics of emoji, where emoji fit in the taxonomy of visual communication, and why this matters. And finally, we talk about the difficulties of translating emoji when it’s not really meant to be a language. Joining us for this conversation are Fred Benenson, an early employee at Kickstarter who built their data team. He’s also infamous for kickstarting a project to translate “Moby Dick” entirely into emoji. Also joining us is Jenny Lee, former New York Times reporter, who is a member of the Unicode Subcommittee on emoji and who recently led the effort to get the dumpling emoji, which is where we start the conversation.

      Emoji basics

      Jenny: I wasn’t a really big emoji user. In fact, the first time I ever heard of emoji was when Fred started his Kickstarter called “Emoji Dick.” And I was like, “What the fuck are emoji?”

      Sonal: What is “Emoji Dick?”

      Jenny: This was before they showed up on our iPhone with, like, perky little yellow faces. I was like, what? It’s, like — sounds [like] something very bizarre.

      Sonal: I just started. I didn’t even actually — just to be blunt, I had a very hard time using emoji, because I didn’t quite understand how to even, frankly, use them. I don’t understand when people send it to me, if it’s not the obvious heart, you know, etc. But as I’ve been using it more, I’ve found myself, sort of, expressing myself now in, kind of, quirky ways. And I don’t know if people really get it or not, but I’m getting a kick out of it.

      Jenny: But that’s the fun of the ambiguity.

      Fred: I have a friend who showed an exchange between a friend of his who was dating a guy, and he would only send her emoji. And she was like, “I just can’t — I can’t handle this.” And he showed me the screenshots of their exchange and it was hilarious.

      Sonal: You’re helping translate.

      Fred: But yeah, and so, like, I was like, “Oh this is…”

      Sonal: You’re like the Cyrano de Bergerac of, like, emoji.

      Fred: Yeah. I was like, “This is what this means.”

      Jenny: I can definitely see it being, like, sort of an irreconcilable difference between people in relationships.

      Fred: Significant other.

      Jenny: Fast forward many, many years, emoji have shown up on our iPhone. And I’m texting with my friend, Yiying Lou, who’s best known as the designer of the Twitter “fail whale.” So, we’re texting back and forth about, like, dumplings. And so I sent her a picture of the dumplings I’m making. And then she texts me back knife and fork, knife and fork, yum, yum, yum, yum, yum. And she goes, “Wait, Apple doesn’t have a dumpling emoji.” I was like, “How could that be?” I was, like, because there’s so many obscure Japanese food emojis, since emoji are from Japan. Like, you have, you know, everything ranging from ramen to curry rice, to tempura, to, like, you know, the rice thingies on a stick to even — there’s even, like  — triangle rice bottle, looks like it had a bikini wax.

      Fred: There’s also the fish cake, which is the white one with the purple swirl.

      Jenny: Yeah, yeah, the spiral. Totally, right?

      Sonal: Oh my God.

      Jenny: And I was, like, how could there not be dumplings, right? Because it’s such a universal food, right? Because there’s, like, pierogies in Poland, and momos, and gyoza, and empanadas. Like it’s just, like, a food from around the world.

      Sonal: I mean, technically, samosa is a dumpling.

      Jenny: Yeah, samosa, ravioli. And I was like, okay, emoji are universal, and then dumplings are universal. How could there not be a dumpling emoji? And just — in my mind, I was just, like, clearly, whatever system in place has failed.

      Sonal: How do you solve a problem like the dumpling emoji?

      Jenny: Yeah, and I found out that emoji are regulated by the Unicode Consortium, which is a nonprofit organization based in Mountain View, California. It now has 12 full voting members that pay $18,000 a year just to vote on issues, including, like, emoji and other kinds of, like, technical…

      Sonal: Are those members in Mountain View or from around the world?

      Jenny: So of those 12, 9 are U.S. multinational tech companies — Oracle, IBM, Google, Yahoo, Adobe, Facebook, Microsoft, and Symantec. Then of the other three full voting members, one is a German software company, SAP. Another is the Chinese telecom company Huawei. And the last is the government of Oman.

      Sonal: That’s a really interesting crew.

      Jenny: Isn’t it an interesting crew? And they have these quarterly meetings, and then I just show up. And they’re, you know, very welcoming. You know, they’re like, you know, “Thank you for coming. What brings you here? Tell us about yourself.” It felt like showing up at church — like a new church. You’re a new member. They all knew each other very well. They’re very excited that there’s, like, someone, you know, young and, like, diverse, who’s just, like, randomly showing up. And so I in that process learn how you get emoji passed, and how they’re regulated. And so, in January of 2016, we submitted a full proposal for dumplings, take-out box, chopsticks, and fortune cookies and got those all passed. So, those will be in Unicode 10, which means that — that’s announced in June of 2017. And so, they’ll actually hit your phones several months after that. I was like, wow, billions of keyboards will be impacted by this and…

      Sonal: That’s amazing. Were there other proposals submitted at the time?

      Fred: Oh, there are constantly proposals. There’s this whole process that people like Jenny — some of them make it through.

      Jenny: It’s complicated, yeah. No, if…

      Sonal: It’s a lot of work. It does introduce some good, useful bars actually for making sure quality gets through at some point.

      Fred: Yeah. And to their credit, the Unicode Consortium has an amazing list of emoji criteria, where they say, “Okay, here’s what we’re looking for for emoji. It’s gotta have like, you know, kind of a unique meaning, in that it’s not covered by other stuff, but it also should have, like, you know, some ambiguity. So, it’s not just, like, literally one thing. It could be used in other contexts.”

      Jenny: Also, there’s one of the more interesting rules, which is no celebrities, deities, or logos.

      Fred: Whoa. The Easter Island head is kind of a violation of that one, but that’s got its own story. A couple of years ago, with a big update, the Easter Island head showed up in, like, the back of the travel section of emoji. And I was, like, what is that doing there? Who is traveling to Easter Island so often that they need to use the Easter Island emoji? And it kinda just stuck in my mind. And then I started using it in this, kind of, like, slightly culturally insensitive way to, like, reference some supernatural phenomenon that I didn’t understand, right? Like, if I was in a conversation with somebody and I was just, like, completely flummoxed, I’d just, like, send that one.

      Sonal: Yeah, it’s like your version of Bermuda Triangle or something.

      Fred: Yeah, yeah, I was just like, “Who knows? Stoneface.” Other people use it for, like, stoned, right? Like, there’s lots of combinations in there. The reason why it’s in there is that there’s a statue in downtown Tokyo. I think it’s a Shibuya station that is called Moai, which is a name of just, like — it’s a proper noun of that statue, which was made by an artist that was, like, a reference to [an] original Easter Island head. So, it turns out, Japanese teenagers use this waypoint to meet each other. And so, that’s how it ended up in Japanese cell phones, and that’s why it ended up in emoji. The artist used this inspiration of Easter Island. The interesting twist is that when you look at it on the iPhone, it doesn’t look anything like the statue in Tokyo. At some point, Apple was like, “We’re not gonna make it, like, this Tokyo one. We’re gonna do it [like] the original one.” Android, on the other hand, their Moai emoji looks like the Tokyo station one.

      Sonal: So fascinating. I read a study — I actually included in our newsletter months ago of someone comparing how emojis look on different platforms and how it actually changes meaning, because…

      Jenny: Totally.

      Sonal: …you can actually think you’re sending one thing and you get something else.

      Fred: That’s gonna happen in any system that has standardization. Like, you’re gonna try really hard to make sure people hue to the specification. But, you know, people do their own implementations and things change. In fact, the whole reason why emoji are in Unicode was because you would send your friend an emoji, and then their cell phone would actually just render the incorrect one. It could be so much worse. And the fact that there is a standard means that, like, you only get these, like, weird edge cases.

      Jenny: There are still some interesting vestiges of, like, the different telcos between Apple and Google. One was Docomo and the other one was SOFTEL.

      Fred: SOFTEL.

      Jenny: SOFTEL. So, they’re basically — depending on who their partner was locally, they kind of inherited those generations of emojis. For example, on Apple, “women with bunny ears” is, like, two women dancing in kind of, like, a “let’s party” kind of way with their bunny ears. Whereas on Android, it’s just the headshot of a woman with bunny ears.

      Fred: And it’s referencing this slightly misogynist part of Japanese culture of bunny woman, which is itself a reference to the Playboy bunny.

      Jenny: Oh, right.

      Fred: And so, they were cocktail waitresses working in nightclubs. That made its way into the Japanese set. And so when it came over to America, like, I think Apple must have been like, “Let’s make this a little more fun.”

      Jenny: One of the easiest things actually to get emoji passed is showing that a vendor uses it. Another argument is for completion. This is actually why chopsticks got passed fairly easily, because we had, like, knife and fork, so you need…

      Sonal: Oh, so you need completion of a set.

      Jenny: …completion. So that…

      Sonal: So, it’s actually you can tell a whole story, like, stringing together a bunch of…

      Jenny: No, I just think that it’s, like, they’re engineers…

      Sonal: Right. You can’t have ABCD and skip the D.

      Jenny: Yeah, yeah. Actually, one of the weird issues is that there are red, yellow, green, purple, blue hearts…

      Fred: Hearts. Yeah, yeah.

      Jenny: …but not orange. So one of the big lobbying efforts has been to fill in the orange.

      Sonal: So the case of the Apple bunny ears and the Japanese bunny women — that was a case where there was an intentional translation to, sort of, obscure the cultural reference.

      Jenny: It’s more that…

      Fred: There are just two separate ones, right.

      Jenny: …they’re often — try to map technically the same emoji, but it’s, like, rendered and sort of interpreted differently. They like emoji that can have multiple meanings. You can also just have, like, emoji that have one meaning. But it really has to be a really good one if it’s gonna be one meaning. So for us, the Chinese take-out box, for example — one of the arguments that we made is that, one, it’s an iconic shape. It also symbolizes both an entire cuisine, which is Chinese food, and also a means of eating, which is delivery…

      Fred: Takeout, right.

      Jenny: …and takeout. Right. And so in that one symbol, you get a lot of, sort of, secondary meaning. And with fortune cookies, like, it’s technically a cookie, but it also means, like, mysterious, and the future, and the unknown, and like…

      Sonal: So, like, sort of primary, secondary meaning. One of the criteria for an emoji to get passed is that it has to have a certain element of ambiguity to it.

      Jenny: Well, I think, yeah…

      Fred: I love this. I’ve been thinking about this so much. When I did “Emoji Dick,” it was more of an experiment around crowdsourcing an emoji itself. Like, I wasn’t, like, so much interested in making a formal case that emoji could be a language because it was still so early.

      Jenny: Yeah, it was very early.

      Fred: Could it get there maybe one day? Yeah. But Unicode makes a really good point. They’re like, “Emoji is not a language. It shouldn’t be a language. The value is that it’s ambiguous.” And I’ve really come around to that thinking, and this idea that the charm of sending an emoji is that it can be interpreted in a couple of different ways. And that’s actually why we value it. And I’ll go further and say that — a lot of people ask me why emoji have become so popular. And I think it’s tied to the fact that we now are just inundated with text. We live in a text culture, right? We communicate via text. Our careers are run over email. We read constantly. Everything we do is mediated through almost literal words. And so, emoji represents this kind of reaction to that. And the popularity of emoji, I think, is largely due to the fact that we need some other way of expressing ourselves over text.

      Sonal: If the pipes are so mechanical, like, phones and machine, you no longer have the non-verbal aspects.

      Fred: Absolutely.

      Sonal: So, this is actually replacing sort of this human element of the glimmer in your eye or, like, the blush on your cheek.

      Fred: Or even just…

      Sonal: There’s an emoji that does that.

      Fred: …you think about the amount of signal you get from somebody’s voice on an analog telephone. And when you strip that out and all you’re communicating is, like, LOL, you don’t actually know how sincere that laugh is, or that chuckle, or whatever that person’s trying to convey. And so emoji gives us a much bigger palette to convey this kind of, like, extra, like, limbic meaning that we wanna have in our communications, but we can’t because we’re texting all the time.

      Sonal: So, to break down the taxonomy of figural representation not using literal text. Let’s talk about where emoji fits. We have emoticons, which are, like, a colon and a parenthesis, and that gives you a smiley face. Or, like, a semicolon and a parenthesis and that gives you a wink.

      Jenny: Right. Using punctuation for existing…

      Sonal: Using punctuation is an emoticon.

      Jenny: Is often ASCII-ish.

      Sonal: Right, because it’s got ASCII art as well.

      Fred: And it goes way back. Some of the earliest references to emoticons go back to the 19th century as well, where people…

      Jenny: Oh my God.

      Fred: Yeah, yeah. People were using colons, and dashes, and parentheses to express, like, a wink. It goes way back. It’s important to add in hieroglyphs and iconography. Other humans have had this idea before. Like, the medium and the technology is kind of, like, incidental.

      Sonal: I’m so glad you brought that up, because it’s so important to not get caught up in technology time. Well, technically, technology includes, like, sticks and stones, so that does go back in time. But in the context of this machine web that we live in, then we have emoticons as part of the taxonomy, and then we have emoji. But how would you guys define emoji?

      Jenny: It’s Japanese. Drawing language.

      Sonal: Emoji.

      Jenny: I don’t know how to pronounce [it] in Japanese, but the Chinese — the “emo” is not for emoticon, or emotion or anything. It’s just totally a coincidence.

      Sonal: Wow.

      Fred: It’s hard not to just hue to the Unicode Standard and say it’s the set of icons defined in Unicode that represent objects, and nouns, and actions and…

      Jenny: The way that I explain it to people is, an emoji is a character — an emoji is something you can put in the subject line of an email because it literally is text. So, in the same way that Unicode has, kind of, defined the standard to unify all the graphical representation of different languages throughout the world — and even non-languages, or like, you know, the Wingdings and all of that kind of stuff. Emoji actually slipped into that entire system. So, there is literally what they would call a codepoint assigned to each emoji — or, sorry, not every single one, because now they’re, like, compound emoji. But there are codepoints assigned to emoji, which basically says, “You know, when a computer sees this codepoint, they render it in a certain way.”

      Fred: But it’s important to, kind of, wrap your head around what’s actually happening inside the computer, because the emoji is being sent as text. If your computer supports UTF-8, UTF-16 — that’s just like a standard way for your computer to handle text, whether it’s your phone or your laptop — then it’s being told, “Render this emoji.” But it’s actually up to your computer’s operating system, whether it’s OSX or iOS or Android or whatever, to go fish out a little image and put it on your screen. And so that image is actually controlled by the hardware manufacturer or the software manufacturer. You know, when it’s actually rendered on your screen, the operating system is choosing which image to show you. And those images are actually stored, you know, in the same way that other images are stored on your computer as little PNG files. And so, Apple, you know, puts those on your computer, and your computer chooses to render those, which is why you may get slightly different, you know…

      Sonal: Different interpretation. Right. I’m glad you walked us a little bit — yeah.

      Jenny: And it’s actually really interesting, because recently Facebook just introduced their own emoji and that, like, basically hijack Apple emoji. So, you can turn that on or off, but essentially, they’ll swap out all the ones on the Apple.

      Fred: And Twitter has had their own set for a while and so…

      Sonal: Why is that? Why do these manufacturers care? Yeah.

      Fred: So, there are interesting copyright considerations here. My guess is a lot of those companies are doing it because A, they can afford to make their own set. B, they wanna avoid the legal liability of using Apple’s set.

      Sonal: Apple, right.

      Fred: And C, like, they think they might kind of have some, like, moment of, like, “Hey, did you see Twitter’s new emoji?” Right? And so, you know, these large companies are kind of…

      Sonal: Innovating on emoji.

      Fred: Yeah, yeah. Like, re-innovating and re-illustrating their emoji. And I think, you know — I think Microsoft actually just evolved to a new set, or was it Android? I think it might’ve been Google or Android. They just upgraded to make it seem a little bit more normal. Like, they had gone from, like…

      Jenny: <crosstalk> the terrible blue and white…

      Fred: Yeah, yeah. So…

      Jenny: Or, there was, like, the blobby ones that were terrible.

      Fred: Yeah, I think Google had blobby ones for a while. And now they’re doing somewhat normal ones.

      Jenny: Scariest emoji ever — the Microsoft emoji are, like, blue and gray, and they look like monsters that hide underneath your bed.

      Sonal: Why are they blue and gray? Why do they look like that?

      Fred: I think it’s just an attempt to be, like, different from, like, the yellow skin tone.

      Jenny: Well, also, you have to — part of the original emoji is, you wanted things that were skin tone neutral. So Apple and Google chose yellow, but Microsoft for some reason chose gray.

      Sonal: Oh, gray because I was gonna say, for Hindu, like, blue is — actually not a bad thing to have your skin blue. It’s, like, a God.

      Jenny: The other thing is, if you have your own set of emoji, you can actually start adding to that set without going through Unicode.

      Sonal: Through the Unicode Consortium, right.

      Social and political sensitivities

      Jenny: So, like, a very good example is the gay family emoji, originally, where it’s not actually one emoji. Another one is, like, man, man, kid, kid. That is actually a compound emoji of four characters glued together using something called a “zero-width joiner,” which is basically like an invisible glue. So, if you are sending that emoji to someone else who doesn’t have the ability to render it out, it actually unravels itself into, like, a multiple character. Now, what you’re seeing is a lot of vendors making compound emojis. And, like, actually one of the places where this is being debated for use is the need for a professional female emoji, right? Because one of the big problems right now, on the existing set of women as represented by emoji is, like, there are only, like, really four roles for women to play, compared to men. You know, men, you can be a sleuth or you can be, you know, a policeman. You can be, sort of, a medical worker…

      Fred: Construction worker.

      Jenny: There are all kinds of things. You can even be Santa Claus. But as a woman, the four things you can be as a role are, basically, bride, princess, dancer, Playboy bunny.

      Sonal: Oh my God.

      Jenny: That’s it.

      Sonal: It just goes to show you how the — I mean, of course, this is the politics of human life [playing] out in these systems. I mean, the perfect example I was thinking of is the rifle emoji, and the case of, I believe, Apple, Google, and Facebook. Charlie Warzel at Buzzfeed wrote a really detailed article investigating this, and about how they, sort of, helped suppress — as part of the Unicode Consortium — the rifle emoji.

      Fred: Right. Emoji already has a gun in it, right? And it’s like, okay, so how many more versions of that do we need? And you’re right, it’s absolutely a political topic. I mean, that issue manifests itself in so many other places than emoji. The country flag stuff is super interesting, because that uses kind of what Jenny’s talking about with these compound emojis. Unicode didn’t actually wanna decide which flags were and weren’t an emoji. So what they did…

      Sonal: Right. You’re legitimizing, then, political issues.

      Fred: What they did was they built this kind of, like, meta country system, so that you would actually be pairing these country letter emojis together. So CNN would go together, and then it would be up to your phone to decide if you showed the Chinese flag. They pushed that decision-making — that, like, political decision-making of which flags to support — off to the handset manufacturer so…

      Jenny: Microsoft actually does something weird there.

      Sonal: What do they do?

      Jenny: They don’t show a flag. They show a flag plus the two letters.

      Fred: The two letters.

      Jenny: Microsoft doesn’t render it, like, normally.

      Fred: To the point about politics being kind of embedded in emoji, it’s not just because these are icons that represent the parts of our lives that we feel passionate about. It’s because there’s a finite palette. It’s not like language, where you can only — you know, you can kind of combine, say, whatever you want.

      Sonal: It’s combinatorial. You can take multiple combinations and turn it into whatever you want.

      Fred: Yeah. Language is, like, you get way more degrees of freedom to kind of express yourself. There’s a finite number of food items that are available to go in there. And when you think about the vast, like, multitudes of humanity, whether it’s, you know, people’s relationship status, their sexual orientation, or skin color, it’s like — emoji is never gonna be able to express that. And so, like, how do you contain this thing that’s, like, growing and kind of has to grow as more and more people use it, but also, by definition, has to be a finite list of icons?

      Sonal: Well, how do they handle the skin tone issue? Because one of the things that I noticed is that on Apple — because I use an Android so I didn’t notice this — you can press down on a thumbs-up, for example, and then you can pick among 15 different shades to, like, pick a skin code shade that’s closest to you.

      Jenny: Five and yellow.

      Sonal: Oh, five.

      Jenny: Yeah, it’s based on the…

      Fred: Do you remember the name Fitzpatrick skin tone scale?

      Jenny: Yeah, it’s actually used — it’s the same skin tone system that dermatologists use to categorize.

      Sonal: This reminds a little bit of being a kid, when you had [a] Crayola box. I remember that the only shade you had — there was, like, a nude shade, or, like, a skin tone.

      Fred: Yeah, and nude was always Caucasian.

      Sonal: And I’d use sepia. I remember using sepia to represent my skin color.

      Fred: I mean, there’s a great history about this in — this is gonna sound weird for me to say. But, like, women’s pantyhose, like, had this issue where nude was always considered Caucasian, and people were, like, “This is ridiculous.” It was one of the earliest blind spots of emoji I remember. It was like…

      Sonal: Right. Well, I mean, if you have, like, only white men designing them. Do you remember when Slack — there was this guy who wrote a post about just the brown hand?

      Jenny: Yeah, yeah.

      Fred: Yeah.

      Sonal: And I remember it was so meaningful, because it’s such a minor seemingly arbitrary thing but then it is true. Like, the first time I saw that I could find my skin color in a system, and to be able to use it, was kind of amazing and empowering. And I think there’s something significant about that.

      Fred: I would totally agree. I don’t share your experience as the person on the other side. And so, it’s funny for me because I don’t…

      Jenny: He’s a white male, for those of you who cannot see Fred.

      Fred: Yeah, so for those of you listening, I’m a white guy. I don’t share that, like, sense of identification with the bright, white, like, <crosstalk> index.

      Sonal: Right. You’re like, “That’s not necessarily me.” It’s just, like, a thing.

      Fred: Yeah. And I’m like, it feels odd to opt into that, which speaks to my privilege as a white male where I just like…

      Sonal: No, it’s not just that. If you’re not exposed to it, you’re not exposed to it. The bottom line is if you’re any person of color, you’re always aware of your color, especially if you’re in a context where everyone else is not the same color as you.

      Fred: And so, when I texted my friends who are not white, and I’m like, should I be choosing that one? And I just choose the yellow skin tone.

      Jenny: Yellow.

      Fred: And that’s just like the — I feel way more comfortable with that.

      Sonal: Yeah, yeah.

      Jenny: So, my solution is, I often send four. Like, it’ll be, like, yellow, light, dark, and then, like, the beige one.

      Sonal: Oh, that’s great.

      Jenny: So it’s like — it’s like a Benetton ad in emoji world.

      Sonal: Benetton emoji, that’s fabulous.

      Fred: So now the, kind of, evolution is that we have yellow for, like, all the human face characters, and then you can choose skin tones for some of them. But it doesn’t get at, like, more nuanced issues about, like, cultural and racial identity having to do with facial structure or hairstyle.

      Sonal: Oh, right, the features.

      Fred: And these are…

      Sonal: That’s a great point, actually, because one of the pet peeves I have is when I used to go to foreign countries and look at billboards, it always glorified that aquiline nose, the face structure — whereas there’s a totally different type of face structure in different areas.

      Fred: Emoji probably won’t ever have that amount of, like, customization, and Unicode gets this. And they actually say, like, “We’re adding, like, 60 emoji a year. This is unsustainable. We feel like the future is inline images.” And that, kind of, breaks my heart as, like, kind of a, you know, nerd standardization guy, like, who really appreciates all the hard work that went into Unicode, and the idea that it is a standard. Because if you’re just sending inline images forever, then, like, you know, you have no idea what’s gonna be on the other side and if they can render the image.

      Jenny: So stickers. So Kimoji, for example, Kim Kardashian’s “emoji…”

      Sonal: It’s awesome.

      Jenny: They’re not actually emoji. These are just stickers. They are images that you can text back and forth. But, you know, again, you know, standards — can you put it in the subject line in the email? And those you can’t.

      Sonal: You can’t so therefore, they don’t qualify. So just to go back to the…

      Jenny: They’re not technically emoji.

      Sonal: Right. So then, going back to our hierarchy, we went from emoticon to emoji and now stickers you would define as a…

      Jenny: Stickers. Stickers are basically inline images. I mean, stickers are just images that you can pick from a palette.

      Fred: And I think you can — you know, in certain apps, you can, like, apply a sticker to an image that it, like, sits on top of it. But you’re then in this, kind of, like, proprietary ecosystem of — that’s okay. But, like, you think about the stuff that really works, and the stuff that really changes the future of the web and communication, and it’s all standardized.

      Sonal: It’s all standard — and you’re saying this as a standardization person. Because my friend, Connie, who wrote a wonderful post on the topic of stickers, argues that emoji are very limited for what you need to do, because she feels that you have so much more expression and the ability to convey so much more with stickers than you do with emoji.

      Jenny: Emoji doesn’t preclude the use of stickers. There are some sets of images that are universal enough that should be hard-wired into the operating systems, and basically can be cross-platformed that an iOS device can talk to — you know, Microsoft Windows can talk to, like, an Android device, can talk to your Mac laptop. Like, the fact that — at least you’re not gonna get little square boxes as long as your operating systems are fairly up-to-date.

      Sonal: Well, that goes, then, to your point about why standardization is important, because you’re now giving up that you’re in this proprietary ecosystem like WeChat or Line, and you only have their sticker set. And you can’t always transfer all these stickers across…

      Fred: And also, if you think about the accessibility issues around stickers, right? Like, people using screen readers— they’re not gonna be able to interpret an image. And, like, emoji actually have names. And so, in theory, there’s much better accessibility for emoji for somebody who’s visually impaired, so.

      Jenny: Yeah. Like, for example, last year, Oxford English Dictionary chose “face with tears of joy.”

      Fred: “Face with tears of joy,” yeah.

      Jenny: Which I always thought looked very sad.

      Fred: Yeah, it’s…

      Jenny: You know, the thing with the eyes and it’s, like, bawling. But that’s actually “face [with] tears of joy.” And you know that because, you know, all these emoji have…

      Sonal: They say the label — Oxford put that in there.

      Fred: That was the word of the year.

      Jenny: So, the word of the year was an emoji.

      Fred: Part of the reason they chose that was that it ended up as number one on my friend’s site, called emojitracker.com.

      Sonal: Oh, right. That’s right. The emoji tracker, which tracks all the use of emoji on Twitter.

      Fred: And for a while, it was just, like — it was, like, the heart emoji or something, or just the smiling face emoji. So, I think it’s really interesting when the top emoji shuffle, because, you know, whenever you start texting with somebody who hasn’t used emoji before, they’re, like, choosing, like, the safest ones.

      Bitmoji and expressing emotion in text

      Sonal: Going back to this idea of some of the companies owning their own emoji, and some of the proprietary open tension between standardization, freedom of expression — what do you make of this notion that part of what we’re doing here is essentially also creating a more machine-readable web, in terms of emotional reading? Because, essentially, you’re now adding a whole new layer where you can codify people’s emotion, sentiment — in ways beyond just a black and white, like, don’t like.

      Fred: I’ve been thinking about this so much, actually, and not in the context of emoji, but actually Facebook reactions.

      Sonal: Yeah, me too. I used to assign and edit op-eds on this topic because I was very obsessed with it.

      Fred: I think it’s a really interesting topic because if you look at traditional sentiment analysis in the data world, it’s kind of a joke. You have to have training data, you have to know good cases. And the…

      Sonal: And just to interject for a moment, as someone who’s tested a million of those systems and can never find one that actually works for my needs, they are so binary. You don’t get anything useful, and you’re not getting insight.

      Fred: One of the reasons there is that words have these degrees of freedom. They can be used sarcastically, and you would never know it based on the semantics. And so, traditional sentiment analysis is really broken, because you’re using these, kind of, like, stale, rigid semantic definitions. What’s really interesting about Facebook reactions is, you know, you think you’re saying, “I love this thing,” or, “I’m sad about this,” or, “I’m angry about this.” But what you’re actually doing, in conjunction with that, is giving Facebook really great labeled data for sentiment analysis.

      Sonal: That’s right. Machine-readable data. That is a holy grail of emotional sentiment understanding. When I was at WIRED, I assigned a piece to a sociologist, Evan Selinger, because I wanted to coin this phrase — the mood graph — because we have an interest graph, social graph, you know, all kinds of other graphs that link all these nodes and ideas. And now, to have, like, a mood graph, to essentially be able to put your pulse on someone’s mood — something very finite, yet constantly changing. It’s just a fascinating thing to be able to codify this.

      Jenny: The sentiment stuff generally correlates very strongly with [the] human face and body. So I think this is also why people agitate so much for emoji that look like themselves. Like the redheads, and people with beards, and people, you know, who are bald.

      Sonal: Or anyone who has curly hair. People with curly hair relate to other people with curly hair.

      Jenny: And so, I think people really love seeing themselves represented in emoji, which is why Bitmoji, which is highly, highly, highly customized stickers in sort of emoji spirit.,,

      Sonal: Oh, my cousins and I Bitmoji on WhatsApp all the time. I think there’s something really symbolically important about Bitmoji, because you are putting yourself in it and conveying in this sticker form. The fact that Snapchat bought it I think is really telling.

      Jenny: Oh, yeah, for $100 million. Is that right?

      Sonal: Right. Especially given that they are changing this culture of how you express yourself through your facial expressions, with face swapping and filters. Connie and I made the argument that it’s sort of like a new — like selfies. It’s selfies as a form of stickers. So what we’re talking about, with the machine-readable, is a little [more] distinct than this, but it’s sort of an interesting idea all the same.

      Fred: I also think it ties into this slightly dubious notion of the uncanny valley where if you wanna try to represent yourself and you wanna have, like, configurability around that, it needs to be, kind of, cartoonish for it to be believable. I think what we’re seeing with Snapchat filters — and I don’t know if you guys have played with SNOW yet. That’s like a…

      Sonal: No, I haven’t.

      Fred: It’s, like, take Snapchat filters and just multiply them by a thousand. It’s just, like, amazing amounts of diversity around the amount of stuff you can put on your face. It is this weird convergence on identity and emoji that’s kind of happening.

      Sonal: I agree and, in fact — this is gonna sound, like, a little out of left field for a moment — but the whole notion around the Chewbacca mask lady, when — you know, that was the most popular Facebook live video ever. It got, like, unprecedented views, and it was simply a woman who was trying on her Chewbacca mask in the car. And she’s laughing and giggling about it. And then she puts her mask on and then she takes it off, and she laughs so uninhibitedly, it’s insane. And I make the argument that what was so empowering — because it took off for obvious reasons — is not the fact that she was laughing so uninhibitedly. It’s the fact that it took putting on and then taking off the mask for her to do that, which is not unlike what happens with communication through these filters, and being able to now express yourself through these cartoon-like ways in a real way.

      Fred: I mean, honestly, it takes me back to, like, theater, and, like, Shakespeare in, like, seventh and eighth grade. I remember having these, like, really intense discussions about, like, what it is to put on a mask and what a mask represents about yourself.

      Sonal: It’s a very Campbellian idea, right? The Joseph Campbell, like, mask, and the myth, and the man. You’re right. There’s a theater — I mean, that’s why people say improv is so interesting for any career field, but I think that there is an interesting moment now coming together with selfie stickers, emoji, Bitmojis altogether, where we do have this new emotional web coming together.

      Fred: Right. And using emoji — the first time I thought about this — could be kind of, like, putting on a mask over your, you know, self to…

      Sonal: Words.

      Fred: Yeah, over your words to convey to yourself this, like, this extra this kind of additional layer, this emphasis of your emotion that you otherwise might not get.

      Creating “Emoji Dick”

      Sonal: Okay. So, going back to you writing an entire book in emoji, and yet you were saying that you’ve kind of evolved in your thinking, you know, that emoji is not necessarily a language but clearly, it is a visual language. And it is a tool for communication. It’s not complete. So, how did you translate that? I mean, what were some of the trade-offs and decisions you made? And, by the way, for the audience — that book was, like, 2009 or that was, like, many years ago.

      Fred: Okay. Okay. I’ll…

      Sonal: So what emojis base were you working off? Did you make them up? Like, what’d you do?

      Fred: So, I’d gotten a text from my college roommate whose wife is Japanese. He sent me an emoji, and I was like, “What is that?” They told me you could download, like, basically a Japanese app and it would, like, awaken your iPhone to the emoji keyboard, like…

      Jenny: Come alive, emoji.

      Fred: It just spoke to me in the, like, you have to hack the iPhone to get the special keyboard of, like, Japanese icons. And I was like, “Oh my God, I want this so bad.” I was like, “This is amazing. I should write a book in emoji.” And I was like, “Oh, that’s a lot of work. I don’t know if I can write a whole book in emoji.” And then I was like, “Or maybe I can translate a book in emoji.” I was like, “Okay, what books would work?” And I was like, “Well, it has to be in the public domain,” because I worked a lot in, like, the copyright reform space. Nobody is gonna just, like, let me translate their book into emoji without a lot of effort. For a moment I thought about the Bible. And I was, like, that’s too obvious. What’s, like, totally, even more inappropriate?

      Sonal: So “Moby Dick” came to mind.

      Fred: Yeah, it came to mind as, like, this, like, impossible book to put into these symbolic characters. As soon as I thought I was like, “No, I can’t do that. That’s crazy.” And I was like, “That’s, like, too hard.”

      Sonal: Honestly, it’s a little bit like — I just came back from seeing “Hamilton.” And so, it’s a little bit like the idea of putting a rap to, like, the founding fathers. That’s what I find so fascinating.

      Fred: Yeah, I would say “Hamilton” was probably…

      Sonal: It’s like a mashup of mediums, and time, and culture.

      Fred: And it’s like one of those things where you tell it to somebody and they’re like, “You can’t do that. That’s crazy.” And then you’re like, “Well, the fact that you just said that made me wanna do it.” And so…

      Jenny: And not only that. There are not one but two whale emoji. Were there at that time?

      Fred: No, there was only the original…

      Jenny: The cute one?

      Fred: The cute one. The, kind of, <inaudible> style one.

      Jenny: Cute one, aww. So, there was a whale emoji.

      Sonal: What’s his name? Ahab is battling the cute whale. Aww.

      Fred: Yeah.

      Jenny: That’s the second one.

      Fred: Yeah, I think it’s called sperm whale — didn’t come up until later. So, I was like, okay, wow. That would be really interesting to do all of “Moby Dick,” because it’s also, like, really long. I mean, it’s 10,000 sentences. And okay, well, if I don’t wanna do this, maybe I can hire somebody to do this. And I was, like, experimenting with Mechanical Turk at the same time. I think it was, like, one of the original Amazon Web Services. It was, like, it would later become, you know, part of that AWS umbrella.

      Sonal: Yeah, I remember people using it for research and stuff.

      Fred: Right. It’s still used for research. It’s still invaluable for that. But, you know, a couple of other people had done, like, an experiment here or there, like, using it, like, off-label. I had made a task at Mechanical Turk — just to ask Turk workers, “If you could ask anyone, like, to do anything on Mechanical Turk, what would you have them do?” And they came up with this long list of stuff. And I don’t think “translate a book into emoji” was one of them. But there was some creativity out there. I was, like, okay, I’m gonna try this thing where I’m gonna hire people to translate “Moby Dick” into emoji, some portion of it, and see if this works. So, I did the first chapter and the results came back, and they were hilarious. They were so good.

      Sonal: They were good.

      Fred: Yeah, they were great.

      Sonal: How did you assess that? First of all, what do you mean you did the first chapter? Like, did they break it down word by word? How do you capture that in emoji? Is it like a sentence?

      Fred: So, I decided I was gonna do it on a per sentence basis. And that actually turned out to be one of the challenging parts of the project was, like — splicing sentences is actually like, kind of, like, a classically hard and a natural language processing problem. And so I kinda, like, figured out a hack to, like, chop it up. And I wrote a lot of regular expressions to basically get the whole book into sentences.

      Sonal: Wow. But you decided basically the sentence was the unit of analysis, not a phrase, not a word, a sentence.

      Fred: You would have a sentence in the task and you’d say, “Pick any of these emoji.” And then I actually wrote my own little emoji picker, because these things didn’t exist at the time. I had gotten the emoji from a friend. He had reverse engineered the iPhone SDK and basically hacked out the PNG files from the software kit to basically have the raw emoji in image form. And so, I took that and just made, like, a little JavaScript, like, HTML thing and, you know, dumped that into Mechanical Turk. And it came back and I was like, hey, this works. And so, I think the sentence that’s kind of, like, on the cover of the book if you go to the website, it’s, like…

      Sonal: The website being Emoji Dick.

      Fred: emojidick.com. “Call me Ishmael” is the first sentence of “Moby Dick.” And the emoji that the Turk worker chose was, like, telephone, man with face, sailboat, whale emoji. It’s perfect.

      Sonal: That’s amazing.

      Fred: That was just like — but the rest of it was just, like, indecipherable emoji nonsense. And some of the people were just, like, all right, “Give me my five cents. I’m gonna click some random emoji.” And other people just, like, clicked every single emoji. So, the plan became — have people translate the same sentence multiple times. So, you get three different emoji translations for one sentence. And then have another set of tasks where people vote on the best, most appropriate translation. So, like, of the three, which one got the meaning across the best? And I was, like, oh, I was just, like, getting really excited about this. And I started doing the math on how much it was gonna cost. And it was, like, oh, it’s gonna be thousands and thousands of dollars. That summer, I met the Kickstarter guys. I started talking with Andy Baio. He was like, “You should put it on Kickstarter.” So that night I went home and put it on Kickstarter, launched it the next day, and ended up working for them and…

      Sonal: And, by the way, how much money did the campaign make?

      Fred: My goal was, like, $3,500. I ended up raising $3,700. So I worked on it for, you know, nights and weekends for another, like, eight or nine months. And then, you know, self-published it on lulu.com. You can still buy it. It gets printed on demand. And, you know…

      Jenny: Do people still buy it?

      Fred: I’ve sold, like, thousands of dollars of “Emoji Dick.” And I’d say hundreds of copies. And probably, like, 500 or 600 copies of it have sold since then, which is not a lot.

      Jenny: I bet this podcast is gonna sell a bunch.

      Fred: Yeah, well…

      Jenny: You better share some of the proceeds with me.

      Fred: Okay, so there are two copies. There’s the black and white copy, which is, like, the easy to print one, and that’s, like, $20 or $30. And then there’s the full color one, which, like, is obviously preferable, because emoji are so colorful. But when you’re printing on demand, 800 pages of color laser hardbound copy — it’s actually really expensive. So, that thing costs, like, $180.

      Sonal: Right, because you’re not printing in bulk.

      Fred: Exactly.

      Sonal: Because you actually save money when you print in bulk, right.

      Fred: So I have to sell that one for that much.

      Sonal: Damn.

      Fred: And, like, people still buy it. In 2013, The Library of Congress contacted me and, you know, they said, “We would like to acquire ‘Emoji Dick’ as our first emoji book.” I was like, “Are you sure?” They’re like, “Yeah, yeah, we’re sure.” And I was telling a friend — and David Gallagher, I think you must know from the Times. And he’s like, you know, everyone submits their stuff to the Library of Congress. It’s not that big of a deal. And I was like, “No, man, they asked for it, like, they’re acquiring it.”

      Sonal: I think it’s a big deal because it was a curatorial point of view.

      Fred: Totally.

      Sonal: They’re saying, “This is a cultural moment. It’s not just a book that was published, and we need to figure out how to acquire it.”

      Fred: I was like, “All right. I’ll spare a copy.” I signed it. I sent it to them. And then they sent me this little, like, you know, certificate in digital form. It was hilarious — and this is my favorite part — is that it’s somehow listed as a translation of “Moby Dick.” So when you look up “Emoji Dick,” it says all these libraries have it, because it’s really just saying that, like, they have a translation. They have the original “Moby Dick.” Now it’s got a life of its own, and people still discover it, and yeah.

      Sonal: That’s amazing. I mean, you actually even curated an art show, didn’t you, based on this?

      Fred: Yeah. Friends of mine put together a kind of emoji survey art show, and there [was] some really great stuff in there. Emojitracker was there. There was a programming language built out of emoji. There was a lot of other good stuff.

      Jenny: I mean, emojis can have their URL. I mean, that’s another thing. They’re literally text, so you can have, like, emoji@…well, I don’t know, @gmail. But you can have emoji in your email address.

      Sonal: Oh, you can?

      Jenny: Totally.

      Fred: You can also buy emoji domains.

      Sonal: So you have an emoji book. You have emoji art shows…

      Jenny: Emoji hackathons.

      Sonal: Emoji hackathons.

      Jenny: So, our big news this week is that in November, in San Francisco, we are going to throw the first-ever Emojicon, which is basically…

      Sonal: What? Is that like Comic-Con?

      Jenny: It’s like Comi-Con, but emoji, of emoji.

      Fred: I really hope people show up dressed in emoji costumes.

      Sonal: I was about to say, I’m gonna show up as — you guys are gonna — Yiying is gonna show up as dumpling emoji for sure.

      Jenny: Or, like, poop emoji or, like, the ghost emoji. So it has many different elements to it. So one is definitely, sort of, this whole “emoji learn” aspect, where it’s, like, panels and talks. And there’s, sort of, emoji film festival, and there’s an emoji hackathon, and then there’s an emoji art show. And then, of course, the opening party emoji where, you know, our goal is to only have food that is also emoji.

      Sonal: So, why a conference? I mean, of course, I see the cultural significance, but to bring people together around this idea of a first-ever Emojicon, like, what’s the significance of that?

      Jenny: Part of it was, I thought it already existed. And to me, the fact it didn’t…

      Sonal: I kind of did too, to be honest. When you just said that, I was like, what?

      Jenny: Yeah. And then I was like, the fact it didn’t exist — and I kind of have this issue where of, like, if I think something needs to be — I will try to make it exist.

      Sonal: You will make it exist, God damn it.

      Jenny: Right. So, we did it with dumpling emoji. We did it with Emojicon. And so we actually have some really cool sponsors. We’re gonna have a lot of, kind of, emoji activists, kind of, out there.

      Sonal: Emoji activists, I love that.

      Jenny: And also, from our perspective — you know, there are a lot of policy decisions around emoji and, obviously, the world really cares about emoji. Whether or not it’s the rifle emoji, or the condom emoji, or, like, professional women emoji. Part of the goal of Emojicon is to open up that discussion, so it does not just held at the Unicode level so to…

      Sonal: So, are Unicode members gonna be attending this conference?

      Jenny: Oh, members of Unicode Emoji Subcommittee including, like, you know, the co-chairs. And we’ve timed it in November between the Unicode Conference itself and the Unicode Technical Committee meeting. And also, like, it’s right around election day.

      Sonal: Well, you guys, thank you for joining the “a16z Podcast.”

      Fred: Thanks for having us. This was so much fun.

      Sonal: This was so much fun. We could keep going…

      Jenny: So much fun, hours and hours on emoji non-stop.

      Sonal: Yeah, I wish we could.

      • Fred Benenson

      • Jennifer 8. Lee

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Software Programs the World

      Marc Andreessen, Ben Horowitz, Scott Kupor, and Sonal Chokshi

      “All of a sudden you can program the world” — it’s the continuation of the software eating the world thesis we put out over five years ago, and of the trajectory of past and current technology shifts. So what are those shifts? What tech trends and platforms do we find most interesting on the heels of raising our fifth fund? Are we just building on and extending existing platforms though, or will there be new platforms; and if so, what will they be? Well, distributed systems for one…

      This episode of the a16z Podcast covers all things distributed systems — encompassing cloud and SaaS; A.I., machine learning, deep learning; and quantum computing — to the role of hardware; future interfaces; and data, big and small. Podcast guests Marc Andreessen and Ben Horowitz (in conversation with Scott Kupor and Sonal Chokshi) also share the one piece of advice from a management and go-to-market perspective that all founders should know. And finally, why simulations matter… and what do we make of our current reality if we are all really living in a simulation as Elon Musk believes?

      Show Notes

      • How advances in hardware and reduced prices are pushing A.I. and other technological advancements [0:27]
      • The current state of A.I. and where it’s headed [8:39]
      • Real-world applications for technology (life sciences, SaaS, and company creation) [20:21]
      • The firm’s philosophy around team-building [32:59] and advice for founders [37:50]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I am Sonal. And I’m here today with a special podcast we have on the heels of announcing our fifth fund for Andreessen Horowitz. And we thought we’d talk more broadly about what’s changed between the first fund and now and, more importantly, some of the technology trends and trends we’re seeing with founders. And to have that conversation with us, we have our co-founders, Marc Andreessen and Ben Horowitz, and our managing partner, Scott Kupor. Welcome, guys.

      Scott: Hey.

      Advances in hardware

      Sonal: Okay. So let’s just kick things off. One of the things that I want to understand is that it’s been — since fund one, which is, what, six, seven years ago?

      Marc: Seven.

      Ben: Seven years.

      Sonal: Yeah, seven years ago. A lot’s changed in seven years, and I’ve actually heard you argue, Marc, that things have accelerated in that time period, more so than previous decades before. So what do you guys think are the biggest shifts now that are important to us in this newest fund, and what changed in that period, like, the biggest things?

      Ben: So, in fund one, when we started, we thought that our timing was really good, despite the fact that I think the world thought our timing was really bad in starting a new venture capital fund. And the reason why we thought that was that there were three gigantic new platforms hitting all at the same time, which was kind of unprecedented in the history of technology. One was mobile, the second was social, and the third was cloud. And that really proved out, through the course of the early history, that the applications on top of those — particularly mobile and cloud — were just spectacular. And I think we’re coming a little bit to the end of the first phase of, you know, some of the obvious applications that could be built on those things, and we’re moving into some new areas.

      Marc: Yeah. So, let me go kinda to the foundations. So, there’s different ways of looking at it. The foundational levels — one is Moore’s law has really flipped, and this actually has happened. I think this actually has happened over the last seven or eight years, actually, almost exactly over the life of the fund. Which is, you know, for many, many years, Moore’s law was a process of the chip industry, bringing out a new chip every year and a half, that was twice as fast as the last one at the same price. And that continued for 40, 50 years, and that’s, by the way, what resulted in everything from mainframes, mini-computers, PCs, and then smartphones. About, you know, 7, 8, 9, 10 years ago, that process actually started to come to an end the way that it had worked up until then. So, chips have kind of topped out at a speed of about three gigahertz, and a lot of people have said, therefore, like, progress in the tech industry is gonna stall out, because the chips aren’t getting faster. I think what’s actually happened is, Moore’s law has now flipped.

      The dynamic now, instead of increased performance, is reduced cost. You now have this dynamic where, every year, a year and a half, chip companies come out with a chip that’s just as fast but half the price. And so, this is the, sort of, just this massive deflationary force, I think, in the technology world, and I actually also suspect in the economy more broadly, where, basically, computing is just becoming free. Basically, what we do in this business is we just kind of chart out the graphs and then just kind of assume, at some point, you’re gonna get to the end state, and the end state is gonna be the chips are gonna be free. Which means chips will be embedded in everything. You’ll be able to use chips for, literally, everything. And we’ve never lived in a world before where you can do that. So, that’s the first one.

      Second one is just the obvious implication from that, which is, all those chips will be on the network, right? So, all those chips will be connected to the internet. They’ll all be on Wi-Fi, or mobile carrier networks, or wired networks, or whatever, but they’ll all fundamentally be on the internet, you know. That’s something that’s now happening at a very rapid pace. And then the third is the continuation of the piece that I wrote, actually, five years ago, which was called “Software Eats the World,” which basically just [says], if you’re gonna live in a world in which there’s gonna be a chip in every physical object, and if you live in a world in which every physical object, therefore, is going to be networked — it’s gonna be smart because it has a chip, and it’s gonna be connected to the network — then basically, you can then program the world. You can basically write software that applies to the entire world. So, you can write software that, all of a sudden, applies to all cars, or you can write software that applies to all, you know, everything flying in the sky, or you can write software that applies to all buildings, or you can write software that applies to, you know, all homes, or all businesses, or whatever, all factories.

      And so, all of a sudden, you can program the world. That’s really just starting, and I think a lot of the — there’s a number of things that make the entrepreneurs we’re seeing these days, in many ways, more interesting and more aggressive than entrepreneurs we’ve seen in the past. And part of it is, they just assume — if there’s something to be done in the world, there must be a way to write software to be able to do it. That’s at a new level of power, sophistication. It’s a new scope of what the tech industry can do. The consequence of that for us, as a fund, is that we find ourselves evaluating business plans and funding companies that are in markets where, I think, seven or eight years ago, we would have never anticipated operating.

      Scott: So, Marc, does that mean that there’s no new innovation in platforms themselves, and everything — all the innovation will be applications that ride on that existing infrastructure, or do you think there’s also the opportunity to build a new platform, even given some of those trends?

      Marc: I think there are new platforms, and I think there will be new platforms. I just think they’ll be different kinds of platforms than we’ve had in the past. The idea of a platform in the tech industry, as you know, up until, you know, 5 or 10 years ago, was there is a new chip that has new capabilities, is faster, and then, therefore, you build a new operating system for it. And that might be Windows, or it might be, you know, might be iOS, or whatever it is. The platforms that we’re seeing getting built these days are distributed systems. So, scale-out systems, sort of, being built on a chip necessarily with new unique capabilities. They are platforms that are getting built across lots of chips. And so, in computer science terms, they’re distributed systems. Cloud is one of the first examples, right? So anybody who uses AWS can now go on and can program an application on AWS that will run across 20,000 computers. And they can run it for an hour, and it’ll cost, you know, 50 bucks. And that’s a kind of platform that did not exist before.

      And, by the way, there are many specific elements to that. So, for example, we’ve seen the rise of, in that category, we’ve seen the rise of Hadoop, and now the rise of Spark for distributed data processing. We’ve seen — in financial technology, we’ve seen the rise of Bitcoin and cryptocurrency, which is, literally, a distributed platform, you know, for currency and for exchanging value. And now, we’re seeing the emergence of a major new platform, which is A.I. — machine learning and deep learning, which is inherently — the great thing about machine learning and deep learning is they’re inherently parallelizable. They can run across many chips, and they get very powerful as you do that. And you can do things in A.I. today as a consequence of being able to run across many chips that you just couldn’t even envision doing 5 or 10 years ago.

      Sonal: So, let’s talk about the rise of the GPU as part of this next platform chip. I mean, I think the biggest surprise people have had is that this is the graphical processor unit, which is something that was developed in the gaming industry for really high-resolution graphics processing, and is now finding, I guess, unexpected — is it a surprise to us that it’s finding uses in these new platforms, like VR, AR, deep learning?

      Marc: It’s actually, interestingly — it’s a new application of an old idea. Back when I was getting started 30 years ago, working in physics labs, if you wanted to run just a normal program, you just buy a normal computer and run the program. But if you wanted to run a program — many physics simulations had this property where you would want to run a very large number of calculations in parallel, right. So, you could basically divide up a problem, simulating anything from a black hole or to different kinds of biological simulations. You could basically write these algorithms in a way that you could basically parcel the problem into many different pieces and then run them all in parallel. There was actually, in the old days, there was actually a whole industry of what were called vector processors, which were, literally, these kind of sidecar computers that you would buy and you would hook up to your main computer, and they would let you run these parallel problems much faster.

      And so, literally, 30 years later, the GPU is — it’s basically a vector processor. It’s basically a sidecar processor that sits along a CPU and runs these parallel problems much faster. And graphics are a natural application of that, but as it turns out, graphics aren’t the only application.

      Ben: Yeah. Actually, interestingly, and I was at a company making one of these called Silicon Graphics — and the applications then were, as Marc was saying, a lot of physics applications, computational fluid dynamics, and simulating, you know, flight simulation, and all these kinds of things that are hard physics to calculate. When you go into the virtual world, and you’re simulating the physics of the real world, guess what? You need the exact same processor to do it. So, it’s a super logical conclusion to what’s been going on, but I think we’re also in the world of big data, seeing kind of more reasons to do just lots of math in parallel. And so, it’s an exciting application.

      Marc: Yeah. You talk about platforms — one of the really interesting hardware platforms that’s emerging right now is Nvidia, which is a very well established public chip company, but very successful, to your point, doing graphics chips for a very long time — has become seemingly overnight — it’s really, of course, the result of years of work — but seemingly overnight has become the market leader in both not just GPUs but also in chips being used for A.I.. And it’s basically extensions of the GPU technology. And we see this overriding theme, which is kind of an amazing thing, which is, basically, every sharp A.I. software entrepreneur that comes in here is now building on top of Nvidia’s chips. Which is, of course, a very different outcome than entrepreneurs of previous years, who would have built other kinds of programs primarily on top of Intel chips.

      Advances in A.I.

      Scott: We’ve mentioned A.I. and machine learning a couple of times here. And one of the interesting things, at least, that I think we see in the industry is, at the same time we’ve got startups doing it, we also see some of the very largest established players investing significantly in A.I. and machine learning. So, certainly Facebook, and Google, Apple, and others are obviously building big operations. How do you think about the universe from an investment perspective? What are the kinds of things that actually lend themselves well to startup opportunities in the A.I. space, versus things that actually might make sense kind of living inside of one of the larger companies, like a Facebook or a Google?

      Ben: Yeah. So, you know, A.I. is extremely broad, and I think one of the challenges that people have with it is they try to paint it as a narrower thing than it is, but one can think of it as an entirely new way to write a computer program. And so, then, it’s applicable to, you know, the universe of problems. So, there are things that advantage a big company. You know, if you’re building A.I. to analyze consumer internet data, like, that’s hard to take Google on at that. They do have an awful lot of data. And you know, Facebook, you know, with A.I., computing power matters and the dataset matters. Having said that, there are a lot of areas where nobody has any data yet, in the areas of healthcare and the areas of autonomy. So, you know, there’s lots and lots of opportunities, and you know, there’s also interesting ideas about, “Well, is there a better user interface than the smartphone using A.I. techniques? And then, what is the form of that?”

      Sonal: What do you mean by that, when you say there’s a better user interface?

      Ben: Well, yeah, if you think about a smartphone, it was kind of an advance over what we used to call the WIMP interface. Windows, icons, what was it?

      Marc: Menus.

      Ben: Menus.

      Sonal: Oh. What was the P?

      Marc: Pointer.

      Ben: Pointer, yeah.

      Sonal: Oh, pointer, right.

      Ben: Which, you know, was, like, a big advance over the text-based interface of DOS. And then, you know, the smartphone with the touch interface, it was more of a direct manipulation — was an advance over that. And so, you go, “Okay, well, but that’s not actually what people do in life,” right? It’s, anthropologically — it’s a backward step, in terms of the natural interface that we’ve become accustomed to, like, for example, natural language. With A.I., you get into a world where things like natural language, and natural gestures, and so forth, become much more plausible. So, there’s, you know, potentially an opportunity to build interfaces for things that you couldn’t before. I mean, I think there’s one, like, really interesting thing, which I’m sure — and I know that Google, and Apple, and all the giant companies are very focused on — which is, how do you replace the current set of user interfaces with it? But there’s another dimension, which is, what are all the applications that you just couldn’t have before, because you couldn’t build a workable user interface for it. And A.I. seems very promising in those areas.

      Sonal: You didn’t mention Amazon, which is sort of the stealth player here, with Echo and Alexa. I mean, really, Trojan Horse of the home.

      Ben: Well, you know, in a way, they’ve got an interesting advantage in that they’re not tied to the last generation of user interfaces, so that they don’t have to pay the strategy tax for shoehorning in their A.I. into, say, the iPhone, and that’s something.

      Marc: Yeah, that’s worth pointing out. There’s sort of two, kind of, classic rules of thumb in this industry. One is for major new advances, especially in things like interfaces, if you don’t own a platform, you can’t do them. And so, the assumption, I think, had been up, until recently, you know, that it would have to be Google or Apple that does these kinds of natural language or interface advances, because they own iOS and Android. The other rule, of course, is the exact opposite rule, which is the one that Ben mentioned, which is the problem that big established companies get into — is what he referred to as the strategy tax, which is, basically, big companies with existing agendas have to, sort of, fit their next thing into their existing agenda, and they often compromise it in the process.

      And so, it’s sort of this ironic twist of fate that Amazon has, all of a sudden, taken the lead from Google and Apple, even though Amazon, you know, famously flopped with their phone, right, which is sort of the obvious place where you have a voice interface. It didn’t matter because they came out with this new product, which was, basically, the speaker, the smart speaker called Echo, and the fact that, all of a sudden, Amazon didn’t have a phone, all of a sudden, became an advantage because they could just do the clean actual breakthrough product without worrying about tying it into the existing strategy.

      Sonal: Right. And those are all still big companies, though. I’m not really hearing where startups can really play in this space, especially when you are describing this huge data network effect that all these big companies have.

      Marc: A year ago, we would have probably been sitting here and say that A.I. was going to be likely would be a domain of big companies, because of this sort of thing of, like, “Okay, only big companies can afford the very large number of engineers that are required to do A.I., only big companies can afford the amount of hardware required to do A.I., and then only big companies can get the giant datasets required to do A.I..” In the last 12 months, what we’ve seen, basically, is all three of those changing very fast, and to the advantage of startups. We’ve seen a lot of A.I. technologies, actually — now, interestingly standardizing — so going to open source. And then the next step is going to be, they’re gonna go to cloud, and that we’re right — because we think we’re right on the verge of that. We think all the major cloud providers are going to be providing A.I. as a service, and they’re gonna really radically reduce the amount of technical knowledge you need to apply A.I.. And so that plays very well to the startups.

      Sonal: So, there will be, like, an AWS for A.I..

      Marc: Yeah, exactly. And that may be literally AWS, or it may be Google, or Microsoft, or all three of them, and you know, in some combination. Or, it may be other, you know, other companies yet to emerge.

      Sonal: An example of the open source, like TensorFlow, Google releasing TensorFlow.

      Marc: Yeah. And this is a big deal, of course. Yeah, that’s right. So, Google open-sourced a pretty significant part of how they do deep learning, and that, actually, now, is something other companies can pick up and use directly. And we see, actually, not only a lot of companies but, like, a lot of university — a lot of student projects now just kind of pick that up and run with it. So, this technology is kind of trickling down very fast.

      Sonal: Just this past weekend, we had a Hackathon. And I think most of the teams had some machine learning, A.I. component into their hacks. And these are college kids.

      Marc: Yeah, yeah. You know, if you’re a 21-year-old junior in college and you’re doing some project, just, kind of — it’s rapidly becoming very obvious that you would have A.I. be part of it, which was very much not the case even 12 months ago. And that’s a direct, to your point, that’s a direct consequence of the open sourcing and kind of this knowledge spreading out. The second thing was the hardware cost, and there, again, the cloud, A.I. in the cloud — just the existence of the cloud is bringing down hardware costs across the board, but A.I. in the cloud is gonna bring that down even further. And by the way, these trends all slam together. So, you get what I think, in a year, is gonna be very common to these sort of A.I. supercomputing chips, with A.I. algorithms in the cloud available to anybody for a dollar, right? And so, there’s gonna be this massive deflation of hardware cost on that side. These big datasets are interesting.

      Ben made the case that the startups can assemble big datasets, and I think that there are, certainly, examples of that. We also see another thing happening, which is the newest generation of experts in deep learning, or many of them are specializing in the idea of deep learning applied against small datasets. If you talk to those folks, what they’ll tell you is — [what] they’ll basically say is — primitive and crude deep learning require big datasets, but the really good stuff doesn’t. Small datasets are fine. And so, that’s still very early, but it’s extremely enticing. It’s an extremely enticing idea, because it really brings a lot of these problems, to your point, further into being tractable for small companies.

      But actually, one of the things you can do with these — especially with these GPUs, is you can literally use the same tools that are used to make video games, and you can create simulated versions of the real world, and then you can actually let the A.I. train inside the simulation. And so, if you’re building a new self-driving car, or a drone, or something like that, you can actually create simulated worlds in which there are everything from earthquakes, to floods, to, you know, thunderstorms, hailstorms. You can create birds, swarms of birds. You can literally simulate the real-world environment, and then you can let the A.I. actually train inside that world. And actually, it’s funny. The A.I. actually has no idea it’s training in the virtual world. It’s learning just the same as if it were learning in the physical world. And so, again, for startups with access to cloud-based A.I., you could potentially run, basically, millions of hours of simulated training at very low costs, and all of a sudden catch up to big companies.

      Ben: Interestingly, you know, the very famous A.I. project that Google did with DeepMind, that whole dataset came from the game playing itself. So, you know, it wasn’t some dataset that Google had collected over 20 years. It was the game playing itself.

      Sonal: So, you guys have both mentioned simulations a few times. Why are they so important? Because I feel like there was this period, like, you know, maybe even a decade ago, where simulations were almost frowned upon as this promised thing that didn’t really actually deliver in what you needed to be able to navigate complex environments in real life.

      Ben: Yeah. Well, it’s interesting, so was A.I. — was frowned upon 10 years ago, saying it was all — it didn’t work. I mean, particularly, neural nets and deep learning were the most frowned upon area. And there’s been similar, kind of, breakthroughs for simulation, first of all. So, if you think about the field of data science and what you do with data, you have a giant set of data, which is always historical in nature, and you can analyze that. And maybe it’s predictive of the future but oftentimes, it’s not. We see this, in particular, in things like really dynamic things, where the past affects the future, like, say, stock picking or the weather, or other kinds of things where data analysis doesn’t get you an accurate answer. Simulation is the flip side of that, where you can say, “Okay, here are all the entities in the world, and let’s generate their behavior over time,” and then their actual behavior feeds back into the simulation, which is critical — you know, a critical component.

      Historically, that’s been difficult at scale, but there have been some really important breakthroughs lately, particularly from a company that we’re invested in called Improbable, which is able to do very large scale scale-out simulation, you know, using cloud computing techniques and some very important new technology that they’ve developed. And so, you can get a really complete picture of the world. And as Marc was saying, you can actually generate your own dataset, rather than collecting it for certain kinds of situations.

      Marc: Yeah. Let me add one thing to that. So, one way to think about it is it’s expensive to make things happen in the real world. Like, it’s expensive to change things in the real world, because the real world is physical, and causing physical changes to happen — I mean, everything from building roads to flying planes, all these things are very expensive. And then things in the real world — changes have serious consequences, right? And so, you know, depending on where you put the dam, or where you put the airport, or what your evacuation plan you have for the city if something bad happens — like, you know, these decisions have huge consequences.

      Ben: Which banks you bail out.

      Marc: Which banks you bail out, which banks you don’t bail out. And so, you always have these consequences, and people who have to make these decisions are often flying blind, because they don’t have any real sense of what’s gonna happen as a consequence of their decisions. In contrast, if you can simulate a world, and if you can run an experiment — if you can simulate the real world or some portion of it, like the highway system, or the banking system, or whatever, and then you can basically introduce change into that simulation, and you can see what the consequences are — it’s very cheap to do that because Moore’s law, the collapse of chips, and the rise of cloud computing, all these other things we’ve been talking about, all of a sudden, make it very cheap to run these simulations. It’s much cheaper to do it in a simulated world, and then there are no consequences. You run a simulation and everything goes, you know, wrong, and everybody dies, or the entire financial system collapses, or whatever. It doesn’t matter. You just erase it and you run it again.

      Sonal: Yeah. You have infinite testability.

      Marc: Great. Yeah.

      Ben: I wanna challenge that. There is Elon Musk’s simulation, in which case, the consequences are quite dire.

      Marc: There is a scenario that we’re all living in a simulation…

      Ben: Right, we’re living in one.

      Marc: …in which case, I would argue it’s gone badly awry, as evidenced by the current political situation.

      Ben: There’s no do-over button in this simulation.

      Marc: Yes. And then you, basically, again, you look at the progress of Moore’s Law and the rise of these new technologies, and you say, “Okay, how about instead of running one simulation, let’s run a million simulations, or let’s run a billion simulations? And let’s try every conceivable thing we can possibly think of, and let’s imagine — let’s literally model all potential future states of the world, and then let’s decide which one of those — which path is the one that leads to the best consequences.” And so we can then make these very big real-world decisions with a lot more foreknowledge of what will unfold afterwards.

      Real-world applications for technology

      Scott: Maybe just to get concrete on some opportunities, what are the other areas in — maybe it’s life sciences, or what are some of the other kind of more tangible areas that you think near-term, as you think about kind of deploying this fund or beyond over the next, you know, 5 or 10 years that might be interesting for, you know, people to think about in the context of real-world applications of this technology?

      Ben: Yeah. So, as Marc was saying, we’re coming into this era of new platforms, and with the intersection of health and computer science, what we’re seeing is really exciting new platforms around data and around, basically, you being able to get much more information about someone’s health from a variety of techniques that had been developed, you know, based on the, kind of, historic breakthroughs and sequencing the genome. But beyond that as well, where we can get really, really powerful data about people and understand them better. And once you have that data about people, wherein you can be predictive of diseases that they might get or things that are wrong, and you aggregate that into a platform, then you can actually make new scientific discovery off it as well. So, that’s one interesting area.

      If you think about the A.I. platform itself, one of the things about it is the hardware that’s been built for it, or that’s been built historically, is for a completely different kind of computer programming. And we’ve seen Google already announce a chip to power their deep learning cloud. And you know, similarly, there’s new breakthroughs in quantum computing, which, at least on the surface, look like they may be very promising for much more powerful deep learning systems, and so forth. So, there’s a lot of things that are coming out of these platforms. And then, you know, as we get to chip and everything, the platforms to run and manage and understand those chips are equally as exciting.

      Sonal: So, you know, one of the themes that’s come up through here is that tech is reaching into places it never did before. I mean, every company is becoming a tech company, or they have tech inside. Or, as Benedict likes to say, “Tech’s outgrowing the tech industry.” The reality is it’s permeating everywhere. And the question I have for us is that we are founded on this thesis that software is eating the world, that’s our premise. And yet we seem to have been making a lot of hard investments, you know, if you count things like Soylent, Oculus, Nutribox. So, are we changing our thesis about hardware as a result of this software eating in the world?

      Ben: No, I don’t think so. I mean, I think that what we see with the companies that you’ve named are interesting. So, Oculus, I think we would all agree that the software component of Oculus is both more complex, has many more people working on it, and is kind of the core of the investment. Sometimes, if you have a breakthrough technology, then you require new hardware to actually support it. And that’s the case there. And I think that Soylent and Nutribox, both of them apply computer science techniques and information technology to get people to optimal health, and that’s what we’re doing there. So, I think we’re big, big believers that, you know, in the last 100 years, the great breakthroughs in knowledge have been the breakthroughs of people like Alan Turing and Claude Shannon, who gave us a new model of the world and how to understand it. And companies that build on that fundamental knowledge breakthrough are what we’re about, and we’ll continue to be about that.

      Marc: Even if some of them may ship their products in a box.

      Ben: Yes, a package is not a technology.

      Scott: Let’s talk a little bit about SaaS. As you’ve probably seen, there’s been actually a bunch of acquisitions in the space recently, but what’s left to do there? So, is the new platform the salesforce.coms and others of the world, or are there actually both, kind of, vertical applications and/or are there other platforms that actually might exist over time in that market?

      Ben: So, there’s SaaS as the metaphorical in-the-cloud version of all the stuff that we had built over the previous, you know, 30, 40 years. So, that’s, like, Workday, Salesforce, SuccessFactors, you know, the kind of big categories. The thing that we believe that’s changed as you go from on-premise to the cloud is, the technology is so much easier to adopt that we’re now seeing software applications for things that you just would never do as a software application, because the cost of — as we used to say in the old days, screwing it in, and paying the army of eccentric consultants to get it going — just wasn’t worth it for, say, expense reporting, which, you know, Concur, of course, built a really powerful product in that.

      But, like, there was no packaged software for expense reporting in the same way that there is now. And I think there’s a gigantic number of categories in everything that you do in business that can be automated in that way. In addition to that, you can scale down to very, very small companies. Companies below thousands of employees never bought Oracle Financials. It would have been insane to do so. But they’re absolutely buying, you know, NetSuite and things like that. And then beyond that, now it becomes economical and very interesting to build vertical applications for industries. So, to build an application that revolutionizes, say, the real estate industry, or something like that, or the construction industry, is becoming extremely viable. And not just as a niche business, but as a real venture capital-based kind of activity.

      Marc: One of the consequences that will be interesting to watch play out is that, historically, enterprise software has been described as represented by companies like Oracle, SAP, IBM. Like, that stuff was really only accessible to the largest companies, the top 500, 1,000 companies in a country. And then, in particular, only in a handful of countries. Those businesses, their revenue and their customer base have always been dominated by, you know, 2,000 or 3,000 companies globally that are these, you know, these giant multinational companies that we’ve all heard of. So, big companies had this sort of inherent advantage versus a lot of midsize and small companies, and then companies in the U.S. and Western Europe had this big advantage versus companies in other parts of the world, where the large companies and the large companies in the U.S. and Western Europe could just afford to make technology investments that small and midsize companies all over the world couldn’t make.

      The sort of changes in SaaS that Ben described, they lead to an interesting conclusion, which is it may actually be interesting for a smaller company, or a company not in the U.S. or Western Europe, to be able to adopt the next generation of SaaS and cloud technology. It’s almost like, the folks who’ve been able to skip landline telephones and just go straight to mobile phones. You can just leapfrog the old stuff because you never had it, and you can just start using the new stuff out of the box. And then the big established companies might have a harder time adapting, because they’ve made these giant investments in the old systems, and it’s hard to just jump to the new thing. And so, there may be a power shift happening from, on the one hand, large companies to small and medium companies that can now more aggressively adopt technology faster — and then from companies in the U.S. and Western Europe to companies all over the world that can also do the exact same thing. And so, at the very least, a leveling of the playing field and possibly even a national shift in balance for small and midsize companies all over the world may all of a sudden get a lot more competitive.

      Scott: So you’ve got, kind of, democratization, on one point. And then, to your point, there’s one version of internationalization, which is adoption across international communities. So, how do you think about, then, the other aspect of internationalization, which is company formation? Should we, then, expect to see more new company formation outside the U.S., partly as a result of some of these trends? And why won’t we see or will we see 50 Silicon Valleys, you know, over the next, you know, 20, 30, 40 years? And how do you all think about what the strategy should be vis-à-vis those opportunities?

      Ben: That would be probably the most amazing thing for the world that could happen in the realm of business and economics. So, we’re hoping for it, and certainly, building — kind of, help trying to build technologies that would facilitate it. And I think the world has never been kind of more ripe for that kind of thing. Having said that, look, there are real network effects, geographical network effects, and Silicon Valley, obviously, has the biggest one in technology. And you always have to keep in mind, and this is something that gets lost, is — there are no local technology companies, right? There’s nobody who sells, you know, internet search to Wyoming. That’s not, like, a viable thing. So, when you’re competing globally, it does matter, you know, “Do you have the best people? Do you have the best executives? Do you have the best engineers? Do you have access to money?” Like, all these things become real competitive things. So, we still are believers in Silicon Valley, and we’re very hopeful that the rest of the world grows and that we can, you know, participate in that as well, but that’s TBD.

      Marc: There’s an interesting macro kind of thing that’s happening. You know, one of the really, kind of, negative stories is that there’s, basically, the world is starved for innovation and growth. One of the data points you point to on that is, there’s now $10 trillion of money being held in government bonds, governments all over the world, trading at what’s called negative yield. This is literally, like, the equivalent of a savings account where, instead of a bank paying you interest, you have to pay the bank interest to hold your money. And so, there’s literally $10 trillion of capital parked around the world that is actually losing money as it sits there, which means people cannot find enough productive places to deploy capital.

      The conventional view, if you just pick up the newspaper and read the economics section, how horrible this is and how it means the world is just starved for growth — the optimistic side of it is there’s $10 trillion of money sitting on the sidelines waiting for something productive to be done with it. What could be productively done with it, right? New kinds of health care, new kinds of education, right, new kinds of consumer products, new kinds of media, new kinds of art, new kinds of science, you know, new kinds of, you know, self-driving cars, new kinds of housing, all these things that need to be done all over the world. And so, the world has never been more ripe for a, you know, very large wave of innovation that would actually be quite easy to finance.

      A lot of the time, you just can’t get things done because you don’t have enough money, right? That’s just kind of the constant state of the world for a very long time. And now, ironically, we live in a world where the opposite is true. There’s actually “too much money.”

      Ben: Yeah, more money than ideas…

      Marc: More money than ideas.

      Ben: …which really can’t be true.

      Marc: It can’t be true, right.

      Ben: You have to unlock the ideas.

      Marc: Human creativity is boundless. And so, if you can get more smart people around the world educated, and with the skills required to do these things, and if you can get them in environments, either create new environments to do that or figuring out how to get more of the people from other places in environments where they can do new things, we could do all kinds of new things, globally. And that’s something that we hope to contribute to, but I think is a very big opportunity for the world.

      Scott: And so, do you think we’re getting to the point where it’s kind of geopolitical risk and rule of law issues that limit adoption or deployment of some of these new technologies in other countries outside the U.S.? It sounds like it’s less so technological advancement.

      Marc: Well, I would say there’s bad news and good news. So, the bad news is, we frequently have delegations of folks coming into the valley from all over the U.S. and all over the world. And they basically come in, and its economic delegations, of different kinds of politicians, or whatever. And they come in, and they’re like, “Okay, what can we do to have our own Silicon Valley?” And then you kind of sit down with them, and you kind of go through, you know, ABCDEF, all these things. “Well, you want rule of law, you want ease of migration, you want ease of trade, you want deep investments in scientific research, you want no non-competes, you want fluid labor laws to let companies very easily both hire and fire, you want the ability for entrepreneurs to be able to start companies very quickly, you want bankruptcy laws that make it very easy to move on and start another company.” And at some point, the visitors give this stricken look on their face, and they’re like, “Whoa.” At the end of it, they’re like, “Okay, but, like, what if we want Silicon Valley but we can’t do any of those things?” And so, that’s the bad news.

      Ben: And they can hire Donald Trump to run their country.

      Marc: It’s ironic that we have this guy running for president who would seriously move us backwards on a number of those topics. So, even we struggle with these things, right? Like, I would argue, the formula is fairly well known. It’s just, people do not want to apply it for reasons that have a lot to do with politics and have a lot to do, you know, with other issues. The good news is it can be done, and then the other good news is it is happening, and there are very, very, very exciting things happening throughout much of the world. There are, you know, very active now startup scenes all through, you know, South America, Brazil, Argentina, Buenos Aires. Amazing things are happening in India. There’s all kinds of startup activity throughout the Middle East. There’s startup activity now throughout Africa. There’s, you know, obviously, China’s been a gigantic success story. Korea has all kinds of interesting things happening. So there are lots and lots of extremely positive early indications of what’s possible in many places all over the world. That said, there are very big political questions about whether or not those founders are gonna be able to operate in an environment that’s willing to let them succeed to the level that they should be capable of doing.

      Ben: A big reason that we raised the fund and are excited about the fund is, it is a backing of our core belief system here, which is, we believe in the creativity, and ingeniousness, and intelligence of human beings and the entrepreneurs that we see and come to Silicon Valley and around the world. And we believe that these people absolutely have the ability to change things, and are changing things. And there’s plenty of room to improve the world, and there’s plenty of ideas to do so. And that’s really what we’re about with Fund V.

      Team-building philosophy

      Scott: So, let’s talk a little bit about, kind of, company-building and founders, in particular. So, you know, undoubtedly, you had a very distinct view of what types of founders you wanted to back when you started the firm, now, seven years ago. How has that evolved, if it all, over time? You know, what has changed either in terms of the types of founders you see, or the types of qualities you see that actually make founders successful, that’s caused you to either augment or rethink some of the initial, you know, foundations for the firm?

      Ben: You know, I think a lot of the things — we had this great advantage when we started the firm that, you know, we, ourselves, were founders. I think that we’ve probably gotten, I would say, more risk-tolerant in our view of founders over time, even though sometimes…

      Sonal: Wait, what do you mean by that? What do you mean by getting more risk-tolerant?

      Ben: Well, we have this thing we say at the firm, which is we’re much more interested in the magnitude of the strength than the number of the weaknesses. We always believe that intellectually. I think that some of the number of weaknesses were fairly terrifying early on, just because, you know, you do have a lot of founders with a very small amount of experience these days, which is also, you know, part of their strength, in that it’s hard to rewrite the world if you’re too steeped in the world.

      And so, I think, over time, we’ve kind of doubled down on that. And really, the founders who have figured out something really important, or who are true geniuses, or have will to power that we can’t even contain in the room — when they bring those things to the table, whatever is wrong with them, we tend to overlook and work with them on that. And if they’re strong enough in those areas, you know, the really interesting thing for us has been those weaknesses do go away pretty quickly. And that’s probably the biggest learning, is I’d say, we went in thinking that, but we’ve gotten even more extreme in our commitment to that kind of philosophy.

      Scott: So almost in financial terms, you’re buying volatility to a certain extent.

      Ben: Well, I think buying volatility, in the sense that we’re buying people who have world-class strengths where we care about them, and regardless of whatever else. There is volatility in that, but you can have a different kind of volatility. You know, you can have people who have gigantic weaknesses that are spectacular without having the strengths. And we’re not trying to buy that kind of volatility.

      Sonal: How do you know, though, that they’re going to be the ones to actually build the companies that scale? Because there seems to be this inflection point, where the very thing that makes you a founder that’s gonna punch through this tough industry, is also the thing that’s pretty much gonna hold you back from really building your company in a really meaningful way if you think you can do everything, you know, your way. And there seems to be an inherent contradiction in that.

      Ben: I think that that would be right if founders did not evolve. So, I think what…

      Marc: And some don’t.

      Ben: And some don’t. And some don’t. Like, some don’t and may get stuck, and they can’t get past that point. But you know, it’s a real common characteristic in great founders that they want to know absolutely everything about the company and how it works, and, you know, every knob and every button. And they really would, like, have a strong desire to actually be able to do every job in that company themselves, if it came down to it. But those kinds of founders also have great ambition, and it’s very logical and easy to understand that there’s never actually been a gigantic long, you know — a really important long-lasting company that had, like, five employees. Those just don’t exist.

      And so, if you’re gonna have to have a bigger company than that, you have to think about the company not only, you know, from the scale perspective, but from the perspective of the people working there. And how are you gonna get great people to work with you if you’re literally making every decision in the company? And I think that not every founder can let go of that, and sometimes it’s a psychological flaw rather than a desire for greatness. And if it’s a psychological flaw that they can’t overcome, then, you know, it’s just like any flaw that any of us have — you know, where we can’t stop eating ice cream or whatever. And you know, there’s nothing we can do at that point. Like, we can give them the logical explanation, but they’ve got to fix themselves.

      Scott: One of the things that we’ve seen even in the short time that the firm has been in business is companies staying private longer, or taking a longer time to IPO. What are some of the implications of that on the company building process? How do you, kind of, balance that new reality, if it is a new reality around how companies stay private, with how you think about building management teams and other issues around the company?

      Ben: I think this gets back to probably one of the more neglected parts of company building, which is, like, “What is the company culture? What does it believe? What’s our way of doing things, you know, when we come to work every day? What does quality mean? How do we prosecute an opportunity, and the kind of philosophy, onboarding, training into that culture, and so forth?” And so you kind of have to develop a philosophy. Like, what kind of employees do you want? How do you want them to behave when they get there? How do people contribute?

      Scott: As we’re getting close to wrapping up here, what would be one piece of advice that you might give either from a management perspective, from a go-to-market perspective? What would be a takeaway for people listening to this podcast?

      Ben: From a management perspective, I think the most common mistake that founders make is, they make decisions based on — management decisions and organizational design decisions — based on very kind of proximate perspective. So, what’s my perspective, what’s the person I’m talking to’s perspective, what’s my HR person’s perspective, without, like, taking the time to go, “Okay, like, how does everybody in the entire company see this decision, and how will they see it once it’s made? Is it motivating people in the way that I think it will? And let’s look past the person I’m talking to feeling good about what I’m saying, and really make this for the long-term health of the organization.”

      Marc: Yep. The single biggest strategic piece of advice we just see across all of our companies is, literally, people just need to raise prices. People need to charge more for their products and services. The good news is you have all these new founders with many different backgrounds who have come in, many of them have never run companies before, run salesforces before. And so they have these extremely sophisticated views on things like products and design and engineering, and then I think, in some cases, relatively naive views on how to actually prosecute a campaign to be able to get the world to use your product. And so, the temptation we see from many founders is to have a one-dimensional view — what I call a one-dimensional view of the relationship between price and volume. Which is, if I price my product cheap, then I sell more of it, because the assumption is just that people just make purchase decisions based on cost. And so, you drive down prices, you drive up volume. And by the way, a lot of the history of the tech industry, like the chip industry, is “drive down prices, drive up volume.”

      But a lot of startups really suffer from having that view. Instead, we encourage companies to adopt what I call, kind of, the two-dimensional view, which is the advantage of raising prices. Actually, there’s a couple of advantages. So, one big advantage — if you raise prices, you can afford a bigger sales and marketing effort. A lot of companies have prices that are actually too low to be able to mount the kind of sales and marketing campaign required to get people to ever actually buy the product. And I call this the “too hungry to eat problem,” right? I’m not selling enough, but I’m not selling enough because I don’t have the sales and marketing coverage required to actually get the product out there, and I don’t have that because I’m charging too little. As a consequence, I’m not selling any despite my low prices.

      The other really interesting thing is that, for a very large number of products, it turns out, if you charge higher prices, the customers take the product more seriously. They impute more value into it when they’re making their purchase decision. And then once they’ve purchased, they’ve made a bigger commitment to it. And in particular, anybody selling anything to businesses, businesses will take something that they had to pay a lot of money for a lot more seriously than something that they didn’t have to pay very much money for. So, you can get a much higher level of engagement and stickiness, and actually use of your product, if you charge more. Going through this, this definitely has felt like swimming upstream for the last several years. We see some glimmers that more folks are starting to figure this out.

      Sonal: Okay. Well, that’s all we have time for. I think this is the first time I’ve actually had all you guys together on the podcast since we did our fifth anniversary podcast a couple of years ago. Kind of amazing how much has changed even in that short amount of time. So, thank you. Thanks, everyone.

      Marc: Thank you, Sonal.

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Ben Horowitz is a cofounder and general partner at the venture capital firm Andreessen Horowitz. He is the author of The Hard Thing About Hard Things and What You Do Is Who You Are.

      • Scott Kupor is an Investing Partner at Andreessen Horowitz where he is also responsible for all operational aspects of running the firm.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      When Humanity Meets AI

      Sonal Chokshi, Fei-Fei Li, and Frank Chen

      Who has the advantage in artificial intelligence — big companies, startups, or academia? Perhaps all three, especially as they work together when it comes to fields like this. One thing is clear though: AI and deep learning is where it’s at. And that’s why this year’s newly anointed Andreessen Horowitz Distinguished Visiting Professor of Computer Science is Fei-Fei Li [who publishes under Li Fei-Fei], associate professor at Stanford University. Bridging entrepreneurs across academia and industry, we began the a16z Professor-in-Residence program just a couple years ago (most recently with Dan Boneh and beginning with Vijay Pande).

      Li is the Director of the Stanford Vision Lab, which focuses on connecting computer vision and human vision; is the Director of the Stanford Artificial Intelligence Lab (SAIL), which was founded in the early 1960s; and directs the new SAIL-Toyota Center for AI Research, which brings together researchers in visual computing, machine learning, robotics, human-computer interactions, intelligent systems, decision making, natural language processing, dynamic modeling, and design to develop “human-centered artificial intelligence” for intelligent vehicles. Li also co-created ImageNet, which forms the basis of the Large Scale Visual Recognition Challenge (ILSVRC) that continually demonstrates drastic advances in machine vision accuracy.

      So why now for AI? Is deep learning “it”… or what comes next? And what happens as AI moves from what Li calls its “in vitro phase” to its “in vivo phase”? Beyond ethical considerations — or celebrating only “geekiness” and “nerdiness” — Li argues we need to inject a stronger humanistic thinking element to design and develop algorithms and AI that can co-habitate with people and in social (including crowded) spaces. All this and more on this episode of the a16z Podcast.

      Show Notes

      • Where AI research is today in terms of hardware and algorithms [0:51]
      • Discussion of creativity and artistic, “generative” intelligence [11:14]
      • Where startups have opportunity in AI [15:51] and a discussion of self-driving cars, including the ethical issues [19:00]
      • How AI needs to learn to interact with humans in a socially-acceptable way [24:24]
      • Adding a humanistic element to AI research to attract more diverse young people [29:11]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and I’m here today with a16z partner, Frank Chen. And we’re interviewing our newest professor in residence. This is actually the third year — the first year we had ViJay Pande, who’s now the general partner on our Bio Fund, and then we have Dan Boneh. And now, we are so pleased to welcome Dr. Fei-Fei Li, who is the director of the Stanford AI Lab, the Stanford Toyota AI Center, and the Stanford Computer Vision Lab, which is pretty much the most important work happening, at least we think.

      Current state of AI

      Frank: AI is the white hot center of both a lot of startup activity, as well as academic research. And, Fei-Fei, why in the world has it gotten so hot again?

      Dr. Li: From my perspective, AI has always been hot. AI is a discipline about 60 years old. In the past 60 years, I call that the “in vitro AI time,” where AI was developed in the laboratories and mostly in research centers. We were laying down the mathematical foundations of AI, we were formulating the questions of AI, and we were testing out the prototypes of AI algorithms. But now, going forward, we’re entering what I call the “AI in vivo time,” [in] which AI is entering real life. So why now? What’s triggering the switch between in vitro to in vivo? I think several things are happening. 

      First is that AI’s techniques have come of age, but what’s driving that? There are two more very important factors. One is the big data contribution to AI. It’s, you know, the information age. The internet age has brought us big data, and now even boosted by just trillions of sensors everywhere. And the third factor that’s contributing [to] this is the hardware — the computing hardware, the advance of the CPUs, of the GPUs, and the computing clusters. So the convergence of, I’d say, mathematical foundations and statistical machine learning tools, the big data, and the hardware has created this historical moment of AI.

      Frank: Why don’t we unpack those in turn? Because I think each one of them in themselves are interesting trends. So why don’t we talk about hardware? We have CPUs, we have GPUs. So it turns out deep learning is great to do on GPUs, because it’s linear algebra and parallelizable. Are we going to see deep learning chips?

      Dr. Li: I think so, and I hope so.

      Sonal: What would deep learning chips look like? Just obviously, much more — the ability to do much more parallelization, but what does that actually look like? Is it like what’s happening with Nvidia’s chips right now, or something different?

      Dr. Li: Nvidia is definitely one of the pioneers in deep learning chips, in the sense [that] their GPUs are highly parallelizable — can handle highly parallelizable operations. And as it turned out, much of the internal operations of a deep learning algorithm, which technically we call neural networks or convolutional neural networks, involves a lot of repeated computation that can be done concurrently. So the GPUs have really contributed a lot in speeding up the contributions, because this can be done in parallel. GPUs are wonderful for training the deep learning algorithms. But I think there is still a lot of space in rapid testing or inference time chips, where it can be used in recognition, you know, in embedded devices. So, I see there is a trend coming up in deep learning chips.

      Sonal: So more specialized hardware dedicated for that.

      Dr. Li: Yeah.

      Frank: Yeah. And we’ve already seen the startups do it like Nervana. Obviously, Google announced the TensorFlow processing unit, right? So they’ve got dedicated silicon as well. So…

      Sonal: So GPU to TPU, basically?

      Frank: Yeah, exactly. Once you know that you’re going to do something over and over again, then you want it in silicon — both for the performance, and then very importantly on the embedded side, for power consumption. Which is — you want your iPhone eventually to be able to do this.

      Dr. Li: But I still think, Frank, that this is a little bit — I wouldn’t say it’s too early, but I think we’re still in the exploratory stage, because the algorithms are not matured enough yet. There’s still a lot of exploration about what to do, in the best way. So, you know, like — this year ICLR, one of the top deep learning conferences — one of its best papers is on [the] particular work coming out of Stanford. Not my lab actually, somebody else’s lab — Professor Bill Dally’s lab — where they’re exploring a sparse algorithm that can enable a specific design of a chip. So, this conjunction of improving algorithms in order to also design the innovative chip is still happening right now.

      Sonal: Is that a new thing?

      Dr. Li: You mean like algorithms driving the design of the chips…

      Sonal: Right, versus the other way around, sort of the chicken egg thing and what comes first.

      Frank: Chip design is already so complicated that you have to do it with algorithms. Humans can’t actually lay out chips.

      Sonal: Oh, I don’t mean algorithm, like a design. I thought what you were saying was, designing the chip for a particular type of almost universal algorithm, which is how I heard that thing.

      Dr. Li: It is designing the chip for a type of algorithm, but it’s a family of algorithms.

      Frank: Your argument is that because we’re not sure what the winning algorithms are going to be, we’re still in this very productive period where we’re trying lots and lots of algorithms. It might be too early to design chips, because to put something in hardware, it’s obviously incredibly expensive to get to an ASIC, right? It’s $50 million to tape out. And so unless you’re sure you know what algorithms are gonna run, you can’t optimize the chips for it. Is that…

      Dr. Li: Oh, actually, I think it’s really important [that] this thing is happening right now. This R&D has to happen concurrently. It’s just, like Sonal said, there’s a chicken and egg dynamic here, that algorithms affect the way chips are designed, but the constraints of the chips could in turn affect the algorithm. I think this is [the] time to explore this. This is the time to devote resources. Of course, in terms of business model, one has to be careful.

      Frank: So the second thing — or another of the three things that you mentioned was that we’ve laid the mathematical foundations for artificial intelligence. And I want to come back to this idea of, look — the hottest thing right now is deep neural networks. But over the 60 years of AI research, we’ve actually used many, many different techniques, right — logical programming. We’ve used planning algorithms, we’ve tried to implement planning algorithms as search algorithms. And so, is deep learning it? Is this what the community has been waiting for or is this just, “Okay, it’s hot now but there’s going to be something else later, too?”

      Dr. Li: I get this question a lot — is deep learning the answer to it all? So, first of all, I’m very happy you actually brought up other algorithms and tools. So, if you look at AI’s development, in the very early Minsky, MacArthur days, they used a lot of, you know, first-order logic and expert systems. And those are very much driven by cognitive designs of rules. But what really, I think, was the first AI spring face is the blossoming of machine learning, statistical machine learning algorithms. We’re looking at, you know, boosting algorithms, Bayesian nets, graphical models, support vector machines, regression algorithms, as well as neural networks. So, that whole period — there is about 20, 30 years of blossoming of machine learning algorithms [that] laid the statistical machine learning foundation to today’s AI. And we shouldn’t overlook that.

      In fact, many, many industry applications today still use some of the most powerful machine learning algorithms that are not <inaudible>. Deep learning is not the newest. It’s actually developed in the ’60s, ’70s by people like Kunihiko Fukushima, then carried out by Geoff Hinton, and Yann LeCun and their colleagues. I think there [are] some really powerful ingredients of the neural network architecture. It is a very high-capacity model that can take almost any function, and they can do end-to-end training that takes data and all the way to the task objective and optimize on that. But is deep learning it? I think there’s quite a few questions [that] remain that would challenge today’s deep learning architecture and hopefully challenge the entire thinking of AI going forward. One of the more obvious one everybody talks about is supervised versus unsupervised training.

      Sonal: And this is I think so important, because a drawback of the current narrative is that it focuses so much on the supervised cases — that we don’t have computers that learn the way children learn.

      Dr. Li: Exactly. First of all, we don’t even know much [about] how children learn. There’s a vast body of education, developmental psychology literature, and that’s not getting into computer science yet. You know, supervised learning is powerful when data can be annotated, but it gets very, very hairy when we want to apply a more realistic training scenario. For example, if one day a company builds a little robot that sends to your home, and you want the robot to adapt to tasks that your family wants to do. The best way of training is probably not to open the head of the robot and put in all the annotated data. You want to just, you know, like show and talk about what tasks there is, and have the robot observe and learn. That kind of training scenario, we cannot do in deep learning yet.

      But there’s more than just supervised training versus unsupervised training. There is also this whole definition of what is being intelligent, right? Task-driven intelligence is really important, especially for industry. You know, tagging pictures, avoiding pedestrians, speech recognition, transcribing speech, carrying goods. Specific task-driven applications are part of AI and [are] important, but there is also the AGI, artificial general intelligence, of reasoning, abstraction, communication, emotional interaction, understanding of intention and purpose, formulation of knowledge, understanding of context — all this is still largely unknown in terms of how we can get it done.

      Creativity and generative intelligence

      Sonal: Where would you put creative AI on that list from — okay, there’s the problems that are yet to be solved — unsupervised, supervised, generalized intelligence, and now also creative intelligence?

      Dr. Li: Actually, you know, here’s one question we should ask ourselves. What is creativity? If you look at the four, five matches of AlphaGo, there were multiple moments when AlphaGo made a movement — Master Lee Sedol was really surprised. And if you look at the Go community, people were just amazed by the kind of creativity AlphaGo has, in terms of making the moves that most people cannot think of. From that point of view, I think we’re already seeing creativity. Part of creativity is just making right decisions in a somewhat unexpected way. That’s already happening.

      Sonal: I’m meant, actually — more interested in the type of creativity where it defies logic, because that’s an example of logical creativity. I’m thinking of something like Jackson Pollock. There is no way a computer is going to waste paint and splatter it, because it’s the most inefficient, irrational thing to possibly do. That’s the kind of creativity I want to know about. I mean, I’m seeing examples of, like, AI-written short films, AI poetry — your own lab, there are people who are writing captions for images. That’s, like, maybe still mechanistic, and Kevin Kelly would even argue that creativity in itself is largely mechanistic, and it’s not as human as we think, anthropomorphic as we think it is — but I really mean like, artistic creativity.

      Dr. Li: Yeah, that’s a great question. So interestingly, you already see some of the deep learning work of transferring artistic style. You can put in a Van Gogh painting and turn a photo into that, but I agree that’s very mimicking.

      Sonal: Mechanistic.

      Dr. Li: Mechanistic. The kind of creativity we’re talking about — blending our logical thinking, emotional thinking, and just, you know, intuitive thinking — and I haven’t seen today’s — any work that builds on the kind of mathematical formulation that would enable that.

      Frank: Yeah, it comes back to one of the three things that you use to set up, “Why is AI winning now?” And that is about data, which is if you’re just going to feed the system a bunch of data and then have the neural net train itself, can that ever lead to something that’s truly creative, which isn’t in the data itself?

      Dr. Li: Right. Exactly. So, this is…

      Sonal: Exactly. Or, maybe it could, by the way, because maybe it can follow the same type of logical arc of history, where you go through a classic phase, a traditionalist phase, an impressionist phase, a post-impressionist phase, an abstract phase. And then you actually go through Jackson Pollock — kind of, Modern Art phase. Like, I almost wonder if you could technically train on that type of history of art and see what happens. I know that’s crazy, and this is completely abstract. And it’s not in any way tied to the actual computer science, but just theoretically.

      Frank: We already have systems that can paint in all of those styles, because there was enough in the data so that it could form a classifier that said, “Here’s the style of Van Gogh,” or, “Here’s the style of an impressionist,” and then we can mimic those styles. So, the question is, down that road, using deep learning, can you ever get to breakthrough new things?

      Sonal: Right. Generative intelligence. Not general, but generative.

      Dr. Li: Generative. So, there’s a lot of thinking on that. We’re pretty far from going from Impressionism to Cubism and all this. But coming back to a more mundane class of work, for example, we are doing computer vision. And some of our work recently is to write a brief captioning or a few caption sentences about images. And then the next thing we did is to start doing Q&A of a picture. And at this point, we start to think, “Can we actually develop algorithms that’s not just learning the training data but learning to learn?”

      Sonal: Exactly.

      Dr. Li: Learning to ask the right question. For example, we just submitted a paper that is — if we show the computer a picture and ask a question about, “What is the woman doing?” —  instead of directly having the computer learn to answer, the computer needs to actually ask a series of questions in order to answer this. So the algorithm needs to — not learning to answer the question directly, but learning to explore the potential space to ask the right question to arrive at the final answer. So the ability of learning to learn is what we want children to have. And this is what we’re exploring in our algorithms.

      Sonal: Okay, so then let’s go back for a moment to something you said earlier, Fei-Fei. You know, I really like how you describe that these phases — the, sort of — the in vitro, like, the laboratory phase, and then the in vivo, like, the in-real-life phase. It’s a wonderful way of clumping the work and the moment we’re at, but there’s always been industry and lab and company, you know, collaboration since the beginning of computing. So, what is different now that startups can play in this space, in vivo?

      Dr. Li: I think several factors. One is that the algorithms are maturing to the point that industry and startups can use it. You know, 20 years ago, it’s only a few top places in the world, top labs in the world, that hold some algorithms that can do some AI tasks. It’s not percolated to the rest of the industry or rest of the world. So, for any startup, or even company, for that matter, to get their hands-on those algorithms is difficult. But there are also other reasons. Because of the blossoming of [the] internet, because of the blossoming of sensing, we now have more use cases. In order to harness data, we need to manage and understand this information. This created a huge need for intelligent algorithms to do that. So, that’s a use case. Because of sensing, we start to get into scenarios like self-driving, and, like, cars. And now suddenly, we need to create intelligent algorithms to have the cars drive. So, that’s what’s creating this, in my opinion, blossoming.

      Frank: The fun thing to watch unfold will be startups versus big established labs and companies. And on the one hand, we’ve got George at comma.ai who built a self-driving car by himself, like, one person. And then on the other side, you’re involved with the SAIL-Toyota Center for AI Research, which is sort of the big industrial approach to this. So, what do you think the relative contributions will be between startups and big organizations?

      Dr. Li: In terms of self-driving cars, who is gonna win the self-driving car competition, right? I think the advantages of the big companies are some of the following. A company like Toyota, as soon as they are committed to this, I hope that they put cameras in their cars. They can already get data very quickly, whereas a startup, this is a lot more difficult.

      Sonal: The data, again, is the big differential.

      Dr. Li: Companies like Google, even though they didn’t have cars at the beginning, they had algorithms. They started this early. So, they now have both data and algorithms.

      Sonal: They were a software company first, as opposed to a car company trying to become a software company.

      Dr. Li: Exactly. The software is such an important part. They actually have an edge there. What about startups? Do they still have an edge? I think there is a lot of business scenarios that might be not so critical on the path for these big companies. But the startup can come in through a more niche area, or more vertical space, and build up their data and algorithm that way. Or, the startup company can do what Mobileye does. Instead of building the entire system — [the] entire car — they build one critical component that’s better than anybody else. And that’s another angle they can come in. 

      Self-driving cars and ethical issues

      Frank: Your colleague, Andrew Ng, who used to be at Stanford and now runs the AI lab at Baidu, has called Tesla’s autopilot system irresponsible, because it got into a crash. Because there are well known scenarios, basically, where the system wouldn’t perform safely. And so, Andrew said, “Look, it’s premature.” So, I wanted to get your thoughts on this, especially since you’re involved with the Toyota program.

      Dr. Li: So, when Tesla’s autopilot came out, I watched some of the YouTube videos. As a mom, I would never want to put my kids or myself into those cars. So, from that point of view, I did, kind of, react — you know, squeamishly on that. But what I’m hoping here, is a really clear communication strategy between the business and the consumers. I don’t have a Tesla, so I don’t know what Tesla told the users. But if the communication is extremely clear about when you should trust the system, and when you should use it, when you shouldn’t — then we get into a situation, you know, when customers are not doing the right thing — who is to blame? And we’re getting more and more into that in AI and ethics — is that, who is to blame? Because every single machine, if used in a wrong way, would have its very scary consequences. I think that’s a societal conversation we need to be having.

      Sonal: Yet another example of how technologists and technology needs marketing. I mean, we tell our company CEOs all the time about the importance of these functions. It just continually reinforces that.

      Frank: Yeah, marketing and training and the right user experience.

      Sonal: Right design.

      Frank: So, this is going to be one of the hardest areas to design for, which is, if we’re on this continuum somewhere between intelligence augmentation and full autonomy, how do you design a system so that the driver knows, “Oh, it’s time for you to pay attention to again, because I don’t know what to do.” Does the steering wheel vibrate? Is there an auditory cue? Like, these are gonna be tricky systems to design.

      Sonal: I agree. And I think this is actually where there is a really important conversation to be had. Nissan has an anthropologist on staff, Dr. Melissa Cefkin. I forgot how to pronounce her last name, but she’s an anthropologist whose full-time job is to study these issues in order to build it into the actual design. And it’s not just, like, software engineers who are designing this. It’s a conversation to be had.

      Dr. Li: In our Stanford-Toyota Center, this center has a group of professors working on different projects. And there is one big project that is led by [the] Human-Computing Interaction Group.

      Sonal: It’s HCI, right?

      Dr. Li: Yeah, it’s HCI because of this.

      Frank: Yeah, it’s great to see, sort of, anthropologists, maybe philosophers come back into the mix, because these complex systems — you’d really want the full 360 degree view of design. It’s not just what technology enables, but what are human expectations around it.

      Dr. Li: And one thing to really keep in mind. Compared to computers, humans are extremely slow computing machines. The information transfer in our brain is very slow compared to transistors and, add on top of that, our motor system — you know, from our brain to our muscles — is even slower. So, when we are talking about human machine interaction and split second decision making, we should really factor in that.

      Frank: Yeah, it sort of brings to mind the famous trolley problem. You knew I was gonna here, Sonal, right? Because I can’t help bringing this up.

      Sonal: And I edited Patrick Lin, who is, like, a long time thinker in this space. And he…

      Frank: Yeah. And the YouTube video that Patrick created is great. So, if you want to sort of see the full exposition, go see his YouTube video. But in summary, the challenge is this — humans are slow. And so, if you get into an accident because your response time was too slow, you’re definitely not liable, right? Like, you just couldn’t control the car breaking in front of you. An autonomous car can actually make a decision. So, imagine that you’re an autonomous car, and then your algorithm needs to decide, “All right, the truck in front of me suddenly braked. I could plow myself into the back of the truck and injure my passengers, or I could swerve to the right and maybe take out the motorcyclist, or I could soar to the left and hit a minivan.” The computer will need to make an explicit decision. And it has the reaction time to actually make an explicit decision. And so, if that decision is explicit, can it be held liable? Can the designer of that algorithm be held liable, because it made an explicit decision rather than having a split second response.

      Sonal: When people bring up the trolley example, it gets really frustrated, because it’s so abstract. But I actually think that the act of going through this thought process is exactly what gets you to answering these questions that you’re asking about the liability — who’s accountable, the emotional tradeoffs that we make, and how to understand even our own limitations, as you point out, Fei-Fei,

      Dr. Li: This actually brings up the topic that in the past few years, I’ve been really advocating in the education and research of AI. We need to inject a strong humanistic thinking element into this, because our technology is more and more in vivo. It’s touching people as real lives. And how do we think and develop and design algorithms that can, you know — hopefully better humans’ lives, but really have to cohabitate with humans. We need that kind of humanistic thinking.

      AI and social interactions

      Sonal: I actually want to ask about a paper that you guys recently just put out. I actually included it in our last newsletter. It was about autonomous cars navigating social spaces. So interesting, because this is lab research in the wild. This is no longer — you know, we can have these algorithms work perfectly fine. But to have them navigate — I’m thinking of streets like in India where, you know, there will be a cow and like 10 buffaloes behind you in the middle of all this, and I don’t know any computer that’s accounting for that. So, I’d love to hear how you guys came to that paper and some of the thinking.

      Dr. Li: This is a project [where] the main PI is Silvio Savrese. It’s the social robot they created called Jackrabbot, so to honor California’s jackrabbit. And the purpose of Jackrabbot is an autonomous driving robot or vehicle that’s taking care of what we call the last miles of driving, where it tends to be in much more social spaces rather than highways. You know, sidewalks, busy cities, campuses, airports, and all this. So, when we look at the problem of last miles of driving, or just the social space, we quickly realize the problem is — you know, not only you have to do everything that a highway driving car needs to be doing to understand the layout of the seeing the pedestrians, the lanes, and all this — you also have to navigate in a way that is courteous and acceptable to people.

      So, one naive solution, people say, “Well, you know, just make a really low speed and stop whenever there’s people.” We tested that. If we do that, the robot will never go anywhere. Because in a very crowded space, there’s always people. If the robot just follows the most naive rule of, “I’ll yield to people all the time,” the robot would just be sitting there from the starting point and not getting anywhere.

      Sonal: Frankly, if that robot was used in San Francisco, it would be kicked, too, probably a couple of times. Maybe people will be really irritated about it — or New York, they’d be irritated in Time Square about it moving so slowly.

      Dr. Li: Yeah, right. So, we thought about that, and we haven’t thought about, you know, what to do yet. We think with — the robot has to have an SOS kind of call. So, what we want to do is to create a robot that understands human social dynamics. So, it can carry it’s on task, for example, going from A to B to deliver something on campus, but do it in a courteous way. So, we started to first record human behavior by data on campus and look at how people gathered together when they talk in small groups, or how they walk — especially, you know, 9:00 [on the] Stanford campus, there’s so many students going into so many classes. But they’re not going in a completely random way. They tend to form interesting patterns, depending on the direction they’re going.

      So, we gather all this data, we feed it into the algorithm. Have the algorithm learn about this — especially from injecting some social rules, such as, people tend to follow others in going the same direction. You do not break two people or several people when they’re talking. So, we injected all these and learned the right way of doing it. And then we put it into the algorithm. And then the algorithm started to learn by itself how to navigate.

      Sonal: Just to probe on that — how to navigate, not how to learn those social cues itself.

      Dr. Li: Right. How to navigate. We give them some social cues, but we only give them high level cues, The detail, for example — the algorithm still has to learn, “When I avoid two people talking, how far do I avoid? Do I avoid them by 10 feet or two feet?” These are the things that are learned just by observing.

      Sonal: Have there been any new surprises yet for you guys, out of this?

      Dr. Li: No. Sorry.

      Frank: When I read the paper, the question that immediately came to mind for me — which is that social norms vary from place to place.

      Sonal: That’s what I was thinking too, the cross cultural aspect, especially.

      Frank: And so when we ship these robots that observe social norms, is this going to be the new localization? In other words, here’s the self-navigating robot, Mumbai edition. Here’s a self-navigating robot, Boston edition.

      Dr. Li: Excellent question. So, my answer to that is, as of now, we have to train them location by location. We have to gather data. But, as I was saying earlier, you know, the next dream I would have is to teach robots how to learn — learning to learn — rather than just to mimic training data. At that point, it should be online learning. It should be incremental learning so that the robot can adapt to different…

      Frank: Right. So you wouldn’t have to train it on a particular city’s actual traffic patterns. You just drop it in there and the robot will figure it out.

      Dr. Li: Exactly.

      Humanistic thinking in AI research

      Sonal: Like the way humans do when you travel like to be — when in Rome, do as the Romans do, so to speak. I mean, I come from the world of developmental psychology, and the development of moral and social mores requires not just a regular cognition, but a metacognition and an awareness of your own thinking — that is a whole new layer that it just complicates things. So it’s super fascinating. Okay, so I want to go back, then, to something you said, Fei-Fei, about this humanistic side of things. Tell us more about what you’re thinking when you say that. Like, do you mean that we should be injecting humanities into computer science, or art — like, you know, I’ve heard of this move from STEM to STEAM. Like, what are you actually talking about when you say that?

      Dr. Li: So, here’s where it all came from. About three years ago, I was thinking — I was observing that in my professional life, there are two crises people tend to talk about, and they seem to be completely disconnected, these two crises. The first crisis is that terminators are coming next door, and AI’s are turning evil, and all this. We’re summoning evil, and AI is gonna just one day rule us all. That’s one crisis. Another crisis we hear here also is about the lack of diversity in STEM, and computing. And from where I stand, the total lack of diversity in AI. And it dawned on me that these two crises are actually connected by a very important hypothesis, which is the lack of humanistic thinking and humanistic mission statement in the education and development of our technology.

      So, let’s look at the first one. Why do we ever think technology might turn evil? Well, technologies are always in the hands of people. Technologies themselves are neutral. You know, be it nuclear weapons or nuclear physics, or just a knife, you know, that can cut [an] apple — you know, in the hands of people, technology can have consequences. So, in order to have responsible and benevolent technology, what we really want is to have a society, have a group of technologists, who have the humanistic awareness and thinking — so that we can use technology responsibly. So, that’s related to the first thing. The second thing is, why are we not — millions and millions and millions of dollars are put into attracting diversity into computing and STEM. And where I stand, I find it very hard to convince women and underrepresented minorities to work in AI.

      Sonal: This is, by the way — despite being at Stanford, which has, what, 50/50 parity in the computer science program with women and men?

      Dr. Li: Oh, no, it’s not 50/50. It’s about 25% to 30%, in undergraduate that we have women. And then this thing just goes down as you…

      Sonal: Oh, goes down as you go higher. Okay.

      Dr. Li: Oh, yeah. The attrition at every stage is grim. And so, looking at Stanford students, they’re extremely talented. Almost any student coming to Stanford, whether it’s an undergrad or a Ph.D, they’re talented enough to be analytical, but also have, you know, great writing skills, care about the world. I suddenly realized here, in our field, as well as Silicon Valley, we’re not sending the right messages to attract people of all walks of life.

      Sonal: What do you mean by that?

      Dr. Li: We tend to just celebrate geekiness, nerdiness. But when you have an ambitious young woman coming into our department, or into the AI lab, she might be thinking about the aging society. She might be thinking about curing cancer. She might be thinking about a lot of socially important topics. If we present ourselves just as geeks loving to do geeky things, we’re missing a huge demography who actually want to turn technology into [a] humanistic mission. So then, suddenly, I realized, we’re missing [a] huge opportunity attracting diversity, because we’re not talking enough or thinking enough of [the] humanistic mission in AI. And that united my two themes I’ve been thinking about.

      Sonal: Just to put a sharp and a point on this. I don’t want to be cliché about “only women and underrepresented minorities would take on ‘the soft problems,’” because there are also other people who might want to take on those challenges of aging, and some of the other interesting shifts that are happening. But to your point, we’re not necessarily inclusive enough — we’re not thinking about this enough, period, regardless of background — to be able to really welcome that type of thinking.

      Dr. Li: I think it’s all walks of life. They come with their experiences and value systems.

      Sonal: That’s fair.

      Dr. Li: The one thing I start to notice. I have a lot of friends who are extremely successful Silicon Valley entrepreneurs and technologists. And, given my own age, all of my friends — many of them are entering the age that they have aging parents.

      Sonal: Yes, this is so top of mind.

      Dr. Li: Suddenly they’re talking about health care.

      Sonal: Which they never did before.

      Dr. Li: When they were [in their] 20s, they’re thinking about beers. You know, they’re not talking about health care. Yeah.

      Sonal: Your point is that having that access to that experience is really important to that perspective.

      Dr. Li: Right. So all walks of life add to our collective thinking and creativity…

      Sonal: Right. It’s a great point.

      Dr. Li: …in our technology.

      Frank: I know one of the things that your lab does is an outreach to high school girls who come to campus for two weeks.

      Dr. Li: This is the brainchild of me and my former student, Dr. Olga Russakovsky. Our hypothesis is, let’s catch girls at the age that they’re starting to think of who they are and what they want to do. And we find the age group of high school freshmen to sophomore thinking about what they want to focus on. So, we created this AI. camp that specifically — we aim for two things. One is, we want to be very technical because we want to get — inspire the future leaders of AI, and talented math and computing students. But we want to attract these students who otherwise might not think of AI, because they didn’t know such a strong humanistic mission is in AI. We actually [ran] a very rigorous hypothesis testing over the summer and wrote a technical paper about this.

      Sonal: I like this approach, by the way, because I get really tired of hearing all the different “camp for this, camp for that, program for this, program for that,” and I feel like, “Come on, guys, are we really solving the problem?” It’s kind of refreshing to hear that you’re taking a much more rigorous approach to it.

      Dr. Li: Right. So our campus designed — in the morning, the students go through rigorous lectures and work with the TA’s and Ph.D students and postdocs on the technical problems of AI. In the afternoon, the girls were divided into four research groups. And each of the research projects is a technical AI project — for example, computer vision or NLP or computational biology. But we put a very strong humanistic statement into each of the projects. For example, last year, we had four projects. The computer vision project uses depth sensors to look at hospital environments and help doctors and nurses to monitor hand hygiene scenarios. The NLP natural language project uses Twitter data during natural disasters, for example, earthquakes to — the girls’ aim is to do the right data mining to find messages that help to do disaster relief. And the self-driving car project, we designed an aging problem of a senior that needs to retrieve drops…

      Sonal: That’s amazing.

      Dr. Li: …and go there and come back. So, everything is very technical, but suddenly they learn that they connect these technologies to humanistic purposes. We have a team of three researchers. Two undergrad, one Ph.D student, and myself — we conducted a rigorous evaluation project on this hypothesis, can humanism increase the interest in AI? And we found a statistically significant difference from the beginning to — before and after for these girls’ thinking. And that particular paper is published in the computer science education conference to show this makes a difference.

      Sonal: That’s great. It’ll be interesting to see what happens when you expand that to other groups.

      Dr. Li: Yeah, we’re running it again this year. And we really hope that this can become a continuous program.

      Sonal: Okay. Well, Fei-Fei, I’m excited to have you join us and bring all these perspectives to our own firm and the entrepreneurs we work with. And we’re so excited. Thank you for joining.

      Dr. Li: Thank you.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Fei-Fei Li

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      Startups and Pendulum Swings Through Ideas, Time, Fame, and Money

      Marc Andreessen and Balaji Srinivasan

      Everything old is new again when it comes to startup ideas and how technology innovation happens. But practically, how does that apply to starting and/or working at startups — especially since the default state of every company is “dying in obscurity”?

      In this episode of the a16z Podcast, Marc Andreessen and 21 co-founder Balaji Srinivasan cover everything from deciding what ideas to work on and the optimal type of startups to work at, to the funding environment and pendulum swings of deciding when to IPO. They also discuss the VC “formula” of weighting product vs. market vs. team; the full-stack approach to cracking industries that tech could never enter before; and recent tech trends and news including The DAO, AI, VR/AR and the “Instagrammification of everything”, more.

      And where does Andreessen stand on the “moral dilemma” of whether entrepreneurs should drop out of college or not? Would Srinivasan still do a PhD today? People’s early career goals should be about maximizing learning skills and minimizing “personal burn”, they argue. But no matter what, Andreessen believes, smart people — from all industries, not just tech — should build things. It’s also easier to get through startup hard times when there’s an ideological mission motivating you, observes Srinivasan.

      This episode is based on a May 2016 conversation that was recorded as part of the Annual Distinguished Speaker Series with Thought Leaders in Technology, hosted by engineering honor society Tau Beta Pi at Stanford University.

      photo credit: Ryan Jae/ The Stanford Daily

      Show Notes

      • How Andreessen Horowitz chooses investments, and balancing team vs. market [0:18]
      • Current trends in the startup space, delaying going public [7:40] and regulatory concerns [12:38]
      • Discussion of new technologies, including Bitcoin [13:35], AI, and VR [20:37]
      • Advice for current college students interested in forming companies [24:57] and an overview of how venture capital evolved [29:12]
      • Questions from the audience [38:51]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. And today’s episode features Marc Andreessen and a16z board partner and co-founder of 21 Balaji Srinivasan. This conversation took place in front of a group of Stanford engineering students as part of the engineering honor society Tau Beta Pi’s Distinguished Annual Speaker Series with Thought Leaders in Technology.

      Balaji: Let’s talk about one of the things I’m sure every student here wants to — or not everyone, but a lot of them, you know, think about startups, think about technology, as an entrepreneur, as a founder, as a potential employee. How should students today, you know, graduating from Stanford think about startups as in both the founder context and an employee context?

      Choosing companies for VC investment

      Marc: Yeah. So the traditional — venture capitalists all have, like, a secret sauce kind of formula of how they think about what they want to fund. And then, it turns out, I think, the formula is all reduced to the same handful of factors, with the exception maybe of Peter Thiel, who has, like, six other factors in his head that he hasn’t told anybody about. But for everybody else, basically, it always reduces down to some combination of market, product, and team. If you talk to people who have been in venture for a long time, what they’ll tell you is, basically, the difference between venture firms, you know, in a lot of ways, is based on how do they rank the importance of market, product, and team.

      You know, as an example, Sequoia was legendary in prioritizing market over team, right? So Don Valentine has — if you go online and Google Don Valentine’s talks, he talks a lot about how the key to success of a startup is to land yourself in a giant market. Like, land yourself in a market that’s about to become explosively large. And, basically, once the startup is in a position where it is the leading company in an explosively large new market, the people become somewhat fungible, like, you can swap the people out. And he would cite Cisco as one of the great case studies of that, which, you know, was actually a Stanford spin-off. A husband-and-wife team, very sharp founders, but they got booted very quickly, actually, by Don Valentine, who brought in professional CEO John Morgridge, who was phenomenal, and actually, you know, built Cisco the company. And so that’s one model.

      The diametrically opposed model is prioritizing team over market. Basically, saying that, you know, the right market or whatever — can you really even know what the good markets are gonna be? Like, how well can you predict? Really what you’re doing is you’re going into business with people, either going to business with really good people or not. If you’re going into business with really good people, one of the things that should make really good people really good is they should be able to find themselves, you know, a good opportunity, right. A lot of startups end up, you know, they succeed based on something different than what they started doing. And so, if you get in business with the right people, they’ll be able to sniff out the opportunity.

      Peter, I don’t want to put words in his mouth, but I think he’d probably prioritize product over market and team, which is — you have to be doing — you have to be making a fundamental advance in technology. He can tolerate a lot of flaws in the people, and he can tolerate a lot of uncertainty around the market. If the product breakthrough is big enough, he’ll make those other bets.

      It’s kind of an angels-dancing-on-the-head-of-a-pin thing. You, kind of — as a VC, you sit around and talk about this a lot. And then if you want an investment company, you kind of figure out some rationalization, I guess, your formula to do it. So, I don’t want to overstate it, but I wanted to go through it, because I do think that is the framework, you know, as you think about startups as either a startup that you might start, or as you think about a startup you might go to. I think that’s a pretty good framework.

      In terms of where — if you’re here as a student, if you’re gonna be graduating — my personal recommendation would be to focus much more on team. And the reason is just because I think we struggle from a distance to evaluate market, and we also actually start to evaluate product. But if you can get yourself in business with really good people, I think, number one — like, if it works, it’s great because those are really good people to be a business with, and they, with you, can build something great. But even if it doesn’t work, even if it’s the wrong market or the wrong product, you’ll still learn so much working with the right people, and you’ll build such a valuable network for whatever you do next.

      It would also apply if you start a company. Like, who do you start the company with? You may end up in a situation where it’s like, do you start the company with a super genius who’s cantankerous and hard to get along with, or do you start the company with the person who’s, like, maybe not quite as incandescently bright but maybe is much more collaborative? And by the way, I don’t know that there’s a right answer. I do know it helps a lot early in your career to be working with really good people, because it really gives you a sense of what good really means and gives you the ability to learn.

      Balaji: Yeah. I would say, one thing that we’ve talked about is that it should be exceptional, at least one dimension.

      Marc: Yes.

      Balaji: It can’t be, like, just pretty good and all these different things. At least one dimension needs to be, like, truly 10x and, you know, amazing to make the bet.

      Marc: That’s exactly right. We talk a lot in our firm about — we have this concept — we say, “We invest in strength, not in lack of weakness.” And again, it’s one of these things that sounds obvious, but it’s proved to us to be a pretty big deal. So, there’s a lot of startups you’ll run into, or you probably have friends who are at these companies or know people at them, and it’s like, team’s good, product’s good, market seems good, they’re making some progress, they’ve got some customers, the customers are pretty happy. Okay. Where is that really gonna go, and where is it really gonna go? Because what’s spectacular about it, right? What’s the thing that’s gonna cause it to jump out from the other hundred, or other thousand companies where you can say the exact same thing?

      So, then you say, “Okay, great. Now, I want to invest in strength. Okay, that’s easy.” The problem with investing in strength, or the problem with running a company, is that the strongest startups — at the point of contact, what you discover is the strongest startups aren’t strong at everything. They’re strong at something, and then they often have — the term we internally use, ironically, is they have hair on them. Which people are always kind of surprised when I start to use that metaphor, but they often have serious team issues.

      Many successful startups have a founder divorce at some point. Like, literally, the founders go to war. And you would think that would be a very bad indicator, and actually, sometimes it’s a really good indicator, because it means that things are really starting to work, and like, it’s time to get serious. And one founder wants to get serious, another one doesn’t, or you’ll have these — some of our best companies are, like, stellar at product and engineering and cannot go get a deal with a customer to save their life, and like, labor for years under the illusion that the way the world works is that, you know, if you have the mousetrap, everybody beats a path to your door, and then three years later, they’re like, “Oh, we have to get salespeople to go sell things.” 

      And so there’s these things, and they’ll just drive you nuts. But if the strength is strong enough, they can really punch through. And so much about this — another thing maybe worth saying is, the default state of every company is just dying in obscurity. And so, so much of that is, how do you punch through? How do you punch through in the minds of the people you’re gonna have to recruit? How do you punch through in the minds of the investors? How do you punch through in the minds of the customers? How do you punch through to the press? Like, how do you actually get yourself visible, such that you can start to attract the kinds of, you know, business, and momentum, and talent, and money that you need to be successful? And so that sort of model of strength versus a lack of weakness I think is pretty important.

      Balaji: Every startup and every project starts as a hallucination, right? Like, it’s a word on a napkin. It literally doesn’t mean anything, and you have to believe it can become much bigger than it is. And always, at every stage, it has to become — you have to believe it’s bigger than it is.

      Marc: Yeah.

      Balaji: Okay, so…

      Marc: That’s right. By the way, it means, in our business, if we’re doing something right, there’s something, basically, horribly wrong with every company we fund. One of the reasons, like, investment banks or the hedge funds don’t just come in and do venture capital is because they’re just horrified at every single investment we do. The one saving grace that we have with that model is, we have a portfolio. So, we get to make, you know, basically 30 grossly irresponsible bets, right, in our portfolio. And then, basically, the math is if we’re doing our job right, 15 work and 15 don’t. And in almost any other area of investing, or any other area of business, if you have that kind of failure rate, right, with that kind of risk level per decision, you would just throw up and go home. If there’s one edge that we have, it’s the ability to kind of indulge in these situations where the strength is crazy, but the weaknesses are also frankly crazy.

      Balaji: Yeah. I mean, like, the thing is, if it gets de-risked all the way, then it’s just a safe investment and there’s very little upside. But I think it also holds for technology, in the sense that, if you read about something in the Wall Street Journal or the New York Times, and technology is on everybody’s lips, it’s probably — not always, but it’s probably started to, you know, have some of the value taken out of it, in the sense that there’s a lot of companies that already built in the space, it’s very competitive, and the technology to look for are often the ones that haven’t got a lot of press yet, you know, that are near inception that are in the labs of places like Stanford.

      Marc: If it’s a buzzword, if it’s something that’s on people’s lips, if there’s magazine articles about it and newspaper articles about it, or, God help us, if it’s on TV, like, the time has passed. Like, we better look for something new.

      Current trends in startups

      Balaji: So related to the subject we just talked about, how people should think about pursuing startups, what does it mean for — so, folks who are, you know, employees, what does it mean when companies stay private longer? And what do you think of the root cause of this relatively new phenomenon, really the last 10, 15 years or so?

      Marc: So, the model for Valley startups, right, used to be very straightforward, which is you’d raise an A round, and then you’d raise a B round to kind of build out your sales force once the product started working. You raise a C round to maybe expand in a couple of other countries, maybe do a little acquisition or something. And then, within, you know, four, five, six years, get to about, you know, 30, 40, 50 million in revenue, and you go public. It was sort of, you know, that was sort of the rite of passage. And then a bunch of things became possible once you were public that weren’t possible before. So, one was liquidity, which is — early investors and employees could start to sell stock. But there are other very important ones. One was, it was viewed as a legitimizing event, especially for companies that sell products to other companies. It was viewed as an event that basically was, you know — a lot of big customers of technology would much prefer to buy technology from public companies, because they feel like they can understand the vendor they’re buying from, whereas these private companies, they don’t know if they’re still gonna be a business or not.

      And then, also, M&A, mergers and acquisitions, you know, it was considered a great virtue of being public — is to have an acquisition currency, right, to be able to issue stocks, and a lot of the great tech acquisitions over the years were done with stock because, you know, you get <inaudible> and go public, and you can use that value to buy things, even if you don’t have the cash. The stereotype is that everybody wants to go to work for a startup in the Valley. I think the reality is, a very large number of people actually don’t want the true early-stage risk. They want to go to a company that’s doing interesting things, but they don’t want to have to, like, go look for another job in six months if something goes wrong, because they’ve got, like, a family. They’ve got, like, a spouse, and they’ve got a mortgage, and they’ve got kids, and they’ve got bills they have to pay. And so there’s actually a lot of talent that got unlocked, once you became public, that you could actually recruit. And so, those were the old days.

      Interestingly, in the U.S., the number of public listed companies in the U.S. peaked in 1997, weirdly enough. And you might think it peaked in, like, 2000 or 2002, or something, but it actually peaked in ’97. And basically, the number of public companies in the U.S. has now dropped by two-thirds since 1997, and that has coincided with a bunch of other things. I mean, one was, you know, we had the stock market crash, and then we had the credit crisis — but it’s also coincided with some other changes. One of the big changes, for example — a lot of tech IPOs actually were individual investors, right. A lot of historical investors and small tech companies were individuals who would read about these things and get excited and invest. If you just look at the statistics on this, the percentage of ownership of tech stocks by individuals has dropped like a rock since 2000. It’s basically now all funds, right, and funds are inherently more conservative than individuals, because funds have, you know, they feel like they have a responsibility to be sober, and so they’re not that excited about the next hot IPO.

      And so, the public market, like, just a lot of the enthusiasm has been drained out of it. The market has changed dramatically. And so, it’s sort of, you know, to Balaji’s question, it’s kind of become in vogue, or in style, to either not go public or at least not go public as fast as before. The good news about staying private longer is that there is something about going public that puts you on a treadmill with quarterly results. They’re like, “Well, you know, I’m not gonna get on this treadmill with quarterly results where I have to hit all these quarterly earnings targets. I’m still gonna be able to do long-term things.” So, the good news about staying private is that you can do these big ambitious projects over long periods of time. And you know, you either get them right or you don’t, but you’re not under any specific quarterly pressure to deliver any particular set of financial results.

      My view is that the pendulum has actually swung too far now in the direction of not going public. Like, too many companies are now staying private too long. It used to be that it was a contrarian view that you should stay private. It’s now become a contrarian view that you should go public. And my argument of why more companies should go public is, at some point, it’s good to not just have all of your results be in the future, but to actually have to deliver in the present. And at some point, it’s good to have an organization that actually, like, knows how to work properly, and knows how to sell things to people, and knows how to, like, have financial plans and hits, and knows how to make money. And it’s all hypothetical until you have to prove it, and I think a lot of companies that are staying private for too long risk getting sloppy and undisciplined. And in the beginning, that’s fine, but at some point, you have to get serious. And if you can go for 10 years without getting serious, I think there’s a real risk that you never get serious. So that’s one.

      And then number two, you know, it’s become massively differentiating to go public, because you get these big advantages. You still can then tap the public markets for more money. People talk about Elon Musk, and you know, SpaceX is still private, but Tesla is a public company. So, Elon Musk puts out this thing, the Tesla Model 3 — the pre-orders, and it gets half a million pre-orders, all of a sudden. Everybody hated Tesla before, because nobody wanted to buy the car. Now, all the investors hate Tesla because, now, there’s too much demand for the car, right, which is apparently equally bad. And so, he just now said he’s gonna do a $2-billion secondary offering, right, in the stock market, and like, even in modern, like, venture capital, it’s hard to raise $2 billion at a shot. Not very many people can do it. And so, he can actually, like, raise that amount of money publicly. He can access debt. And then, you know, you go back to the acquisition currency. Like, we’ve probably been in a slow period for M&A for a while, but there is no question. There’s gonna be a lot of M&A in the years ahead, and the companies that have public currencies, they’re gonna be able to be the acquirers and able to get big and become much more important. So, I think the pendulum is gonna swing back in the other direction. There’s a crop of companies, good companies definitely gonna go public.

      Balaji: I think another part is also Sarb-Ox, and all the rules, and then Dodd-Frank, and so on, has made it quite difficult to be a public company from a compliance perspective, and the fixed cost associated with that.

      Marc: Yeah. So, there’s this thing, Sarbanes-Oxley, which I see somebody in the audience yawning, and this topic is gonna make everybody yawn, and so I’m not gonna go into detail. You can Google it if you really want to learn about it. But it’s the regulatory, kind of, threshold that public companies need to hit on how they deal with risk and do reporting, and all this stuff. And the knock on Sarbanes-Oxley has been exactly what Balaji said, which is it’s basically a burden that falls disproportionately on small companies, because big companies have huge staffs of lawyers and finance experts, and so forth, who can do all this stuff, but small companies, the burden falls directly on the management team.

      Our partner, Ben Horowitz, now argues the opposite side of this, having seen a lot of companies — which he argues, if you’re good enough as an operating team to actually comply with Sarb-Ox, then you’re good enough, basically, to do anything. Like, basically, not everything in it makes sense, but it sets a bar for what it means to be an operating business that’s operating in a responsible way. So, I think he’s actually flipped a little bit on that, and I think he would argue it’s actually part of being a responsible company at some point.

      Future of Bitcoin, AI, and VR

      Balaji: Interesting. It actually kind of gets into our next question. We’re gonna talk about a few important technologies. One thing that I’ve thought a lot about is that the ultimate, kind of, solution to this is gonna be something related to the Bitcoin/Ethereum crowdfunds that are happening now on the internet, where the regulatory stuff has to be worked out about that. But you do have a very large potential pool of capital that people can use for this kind of thing, and that might be, you know — it’s is an essay that Naval and I wrote a couple of years ago about, like, an app coin. So, you’d actually start a company and actually issue a coin that could be used to redeem for calls of that SaaS service. So, that’s one model that might help.

      Marc: You might just mention — this is a whole new model for how to think about, sort of, crowdfunding taken to another level. You might just mention the DAO and what that is.

      Balaji: Yeah. So, this is a pretty interesting concept on where — so Ethereum, it’s something that was based on Bitcoin, initially, and is sort of like a more programmable version of Bitcoin in some ways. There is a thing called the DAO, which raised almost $130 million online in a purely distributed way, just with digital currency, without any stock market or what have you. There’s all kinds of regulatory hair on this animal, and people can pull their money out of it. So, it’s sort of like a VC fund, where the LPs don’t actually commit until they see the first investment. So, I think there’s gonna be all kinds of stuff that happens with it. Nevertheless, I think it’s a very interesting experiment, and something which will probably be relevant for you guys, not this year, not next year, but in maybe 5 to 10 years, in terms of potentially an alternate way to get financing for something. So, actually, that leads us into important technologies, right? So, let’s get a quick riff on them one by one. So, starting with maybe, you know, talk about Bitcoin and blockchain, then FinTech more broadly.

      Marc: Yeah. So I’m gonna turn the first one around. So, Balaji is the founder of one of our two big Bitcoin investments, so.

      Balaji: Sure.

      Marc: Balaji, how’s Bitcoin doing?

      Balaji: How’s Bitcoin doing? Yeah. So, you know, like the Gartner Hype Cycle, right, something we think about a lot. We think of it as this fundamental thing in technology that is — you’ve got this trigger, and then people get really amped about a technology, and everyone’s doing it, oh, you know, bots are at that stage right now. And then you try to actually do it, and you find it’s actually hard, and everyone gets demoralized, and they quit. And you’ve got the trough, and then it’s those guys who stick it out in the trough and pull up over here that, you know, things actually happen. So, that happened with, like, the dot-com bubble. Everyone was hyped about it in 2000, it crashed. And then, actually, you built all these massive businesses. And it happens on, like, larger and larger cycles as well. Carlota Perez — she’s got this whole theory about why that happens. And it, kind of, happens at different scales. And we, sort of, think that’s happening for Bitcoin in the sense of, you know, there’s a huge amount of excitement like 2013, 2014, you know, “Oh, my god, new paradigm.” Then, you know, like, “Oh, the price crashes.” And now it’s coming back up with a lot of, like, micropayment stuff, interesting things happening this year.

      I think the blockchain stuff is actually right at the top of the Gartner Hype Cycle, and I think it’s gonna crash down, like, towards the second half, you know, of this year when people actually try to implement it. That’s where I kind of think Bitcoin and blockchain is, and I would say that, you know, in addition to our kind of point earlier about, like, you know, getting technologies that nobody knows about at all, that are in the lab right now. I think other kinds of technologies to really look at are those that people have written off, right, like, you know, VR after Second Life. And so, that’s the kind of thing to look for — the stuff that people think of as, you know, dead or didn’t work, or what have you, and find out why.

      Marc: It’s actually very funny. You don’t remember the first time VR got written off.

      Balaji: Oh, no, that’s true.

      Marc: You only remember the second time it got written off.

      Balaji: I remember the second time it got — yes, that’s right.

      Marc: No, actually, you remember the third time it got written off. I remember the previous two. It got written off after VPL. It got written off after the VR — there was a whole VR wave in the late ’80s — one of the great all-time hacker movies, “One More Man.” It was kind of a peak of that cycle. And then we bought a VR company, Netscape, in ’95 to do VR/ML, which is VR on the browser. You may note that that didn’t work. And then, right, there was Second Life, which was, like, the third cycle.

      Balaji: Right.

      Marc: One of the things we talk a lot about is, say, two operating principles in how we think about technology. One of the things I’ve come to believe — there are almost no actual new ideas, right. Basically, everything that is gonna be a big deal in the next 30 years is in a lab somewhere, probably here in a lab at Stanford. And so, the eureka moment is, like, an almost non-existent thing. Maybe every once in a while, but there’s almost always a 20- or 30-year backstory of research that often, by the way, turns out to be 50, 60, 80 years backstory of research before something pops. And then the second thing is just, yeah, things take time. There’s this concept called the AI winter, and literally, there have been surges of enthusiasm and crashes in AI. And I think we’ve counted there were, like, 5 AI winters between 1950 and basically 10 years ago.

      Balaji: Even the term AI has only come back recently after neural networks themselves came back, because everyone was like, “Oh, AI is all rule-based, and ML is the new thing.” And [we’re] having another mini-cycle within that where, like, Chris Dixon and I joke that so many AI companies are just a collection of if-else statements. And you know, it’s like, “Okay.”

      Marc: Which are very compelling on first demo.

      Balaji: Very first, yeah, but it’s always on rails, right? And then when you try to get it a little bit off, then it’s like, “Cannot compute. Great.”

      Marc: Yeah. And so I think, Balaji, that’s a very important kind of fundamental point, which is it’s not — I mean, what’s new is important, but it’s often what’s new where there is a track record of intellectual depth that’s gone into it over a long enough period of time that people really have thought hard about it. And it turns out, that track record is almost always multiple decades. And then, whatever happens to be hot or not in any particular moment, is really not predictive of what’s actually going to happen.

      Balaji: Exactly. I think, you know, in particular, there’s two things, if you ask me, you know, what, like, to look at for startup ideas, and so on. So first, I’d say, don’t do a startup unless you’re ideologically driven to make it succeed beyond the economic motivation, because it’s actually very hard. But if you do want to just find startup ideas, there’s this book, “The Sovereign Individual.” It came out in the late ’90s. It’s the most prescient thing in the world. Most bestsellers, you can take the 300 pages and compact them into, like, a one-page summary, and there’s actually websites that do that, right? Whereas, this book is the opposite. You can take, like, a page and turn it into a Ph.D thesis. And what’s awesome about it is, you know, we kind of think Satoshi read through “The Sovereign Individual” and actually made Bitcoin, in part, on that basis, because the description of it is so lucid. But what’s interesting is, there’s other pages of it which haven’t yet been implemented. So it’s like, the “Book of Prophecies,” and you just flip through it, “Oh, let me do that line,” right? So…

      Marc: So then the kicker of, you know, that book ripped off another book, an older book.

      Balaji: What’s that?

      Marc: It’s an older book called “The Twilight of Sovereignty.”

      Balaji: Interesting.

      Marc: Which was written by a guy named Walter Wriston, who was the founder of Citibank, who spent 40 years in banking, 40 years in, like, big New York institutional banking, and his conclusion at the end of it was, it was all bullshit. And he basically wrote a book predicting, basically, the rise of networks and distributed finance, distributed money. This is like 30 years ago.

      Balaji: Yeah. So, I mean, what’s interesting is, a lot of those guys got the general direction right, and then there was some aspect that actually turned out to be much more difficult than they thought. For example, like, autonomous robotics. Well, actually, that’s really hard because of the number of degrees of freedom and the probabilities, but it’s doable with enough training data. I think the other thing that, you know — I think of it like a “Back to the Future” thing that’s very important — is this thing called Tiebout sorting. So, like, a while back, we found this guy who’d done it in 1956, and he had a bunch of assumptions for this model of how people could sort into, like, basically, many governments around the world, and he assumed like, “Okay, you have search. You have perfect information. You have perfect mobility of this, you have that.” And he basically, like, assumed the smartphone. They wouldn’t have put it that way at that time, but 1956, he assumed the smartphone is like, “Oh, wow, you can solve all these problems with governance and so on.” So, like, literally 60 years later, you can go back, you know, dust off this “Raiders of the Lost Ark” stuff and just, you know, go with it, right? And you’ll sound really smart because you can just, like, read off the “Book of Prophecies.”

      Okay. So, other important technologies, all right. So AI, right? We just kind of talked about this a little bit. So, autonomous cars, drones, ML, and software, what is your take on this?

      Marc: Yeah. So, magic is happening, and I think everybody here probably knows this by now, but something has changed. And actually, what that something is, is a matter of some debate, and it’s probably multiple somethings. But an entire battery of techniques that people have known about for a long time, plus some new techniques in machine learning and deep learning have really started to work. 2012 was kind of the tipping point for that. And now it’s really building steam. And then it also feels like something changed —.part of the passage of time in our industry is just Moore’s law, allowing processors to kind of catch up with our ideas, and the rise of this new generation of GPUs that are able to run neural networks and deep learning algorithms is a really big deal. And then, you know, we now have existence proofs of, you know, fully running autonomous cars using deep learning. We’ve got autonomous drones with deep learning. We’ve got, you know, AlphaGo, the great accomplishment that Google recently had, that DeepMind had. Like, significant breakthroughs are happening. I would say something both very dramatic happening, but also something very real happening.

      Balaji: Yep. I would add to that. Actually, just data. Like, because, you know, like, many of these algorithms you just put 10x of data at them and they work, and 1/10 of them don’t. And so, like, just the ease of collecting massive amounts, right?

      Marc: Yeah.

      Balaji: So, VR and AR. So you know, Oculus and Magic Leap, and stuff like that, what are your thoughts on that area?

      Marc: Yeah. So, very exciting. So VR, right, is the idea of the headset that you, basically, are in a completely computer-generated world. I’d like to say, the world’s now divided into two groups of people. People who haven’t tried the shipping consumer version of Oculus, who think VR is stupid, and then people who have tried it, who think it’s the future of everything. And so, if you haven’t tried it, find somebody — they just started shipping. Find somebody who has one and try it. It’s a really profound thing.

      The other idea people are playing with is augmented reality, or AR, which is the idea of — you still see the real world, but you have computer-generated imagery kind of populating it. And there’s a company called Magic Leap in Florida that’s doing this, and Microsoft has a thing. We actually argue there’s two kinds of AR. There’s the kind that people are talking about, because they find VR too scary — and that’s why all the news articles on VR are all, like, very emotionally loaded, because it’s invariably a picture of somebody with this thing strapped to their face, right? You don’t actually get to see what’s inside the VR. You just get to see the idiot sitting there in the chair with, you know, the alien Facehugger, like this, and then everybody thinks it’s funny. To a lot of people who find VR too weird, AR feels like it must be more normal, because I still get to see everybody — and I think it’s actually a little bit of an intellectual crutch for people who just can’t quite come to grips with VR.

      That said, there’s the other form of AR, which is, like — if we can get AR to really work, right, and if we can get to the vision that I think everybody in the industry has, which is — get a pair of, you know, very light eyeglasses or, even better, contact lenses that overlay computer imagery on the real world. Like, that is a big deal. There are teams — there are a handful of companies now that have teams that are super focused on this.

      Balaji: Two thoughts, one on AR and one on VR. One thing that I think about AR is if that kind of thing can work, I think you can have what we think of as, like, the “Instagramification” of many more things, in a sense of, what is Instagram? So, yeah, it’s a photo app, but then it also is something that takes somebody who has no skill in photography and gets them to, like, an eight, because you got a programmer on your shoulder, and you know, he’s like, “Oh, put the f-stop there and whatnot, and don’t generate, and so on.”

      Marc: There’s always at least one filter that makes any photo look good.

      Balaji: Exactly, that’s right. No, I actually think, like, the next version of Instagram will make people prettier, right? Like, I call it Tinder for Instagram. So…

      Marc: Just keep swiping until you get attractive enough.

      Balaji: Well, yeah, exactly. You just got a filter that just morphs it just a little bit, right?

      Marc: It’ll come in handy.

      Balaji: Exactly. The thinking is, though, that Instagramification — you could apply to many other areas with AR, right? Like, so the classic examples are you’re a mechanic, and you put on the glasses, and now, you know, every part lights up, and you see the 3D schematics, and you tap here to order the replacement from Honda, and so on. Or you’re a surgeon and you can actually see the person’s x-ray superimposed on them. And so, it’s like you’ve got a superpower, right, in that sense. Which actually, you know, <inaudible> a while back. 

      And then, on the VR end of things, you know, one thing when people, you know, kind of dismiss VR, I always ask them, “Okay, how much time do you spend looking at a screen? How much time do you spend looking at, like, a laptop or a phone?” And they’ll say, you know, “Okay, maybe, you know, six hours a day.” And so, I’ll say, “Okay, well, that’s like 50% of your waking hours.” And we’re probably gonna replace a significant percentage of monitors with VR, with something to the 2D world, right, and there’s gonna be a new Windows that’s based on the 3D universe, which has totally different GUI metaphors. So, that’s an interesting kind of company to build that doesn’t exist yet. But that company — okay, so when you’re wearing this VR thing to do work, not just to play video games, well, actually, most of your life is in the matrix. So, that’s gonna be kind of interesting in, like, 5 or 10 years. Everyone’s wearing these kind of things.

      Marc: It’s coming.

      Advice for college students

      Balaji: Great. Okay. What should Stanford students be thinking about doing after graduation or, dare I say, instead of graduation? That’s question number one. And then related, what advice would you give if you’re at Stanford right now? And what should a student walking down this hall do right now?

      Marc: Yeah. So, I used to — people used to ask, you know — so, obviously you’ve got — the example is of Mark Zuckerberg, and all these founders who dropped out, and so, therefore, you know, everybody should drop out and start a company. And people used to ask, you know, “Should I stay? Should I drop out? What should I do?” And it used to be a very — I used to feel, like, a real moral challenge answering that question, because I felt like, if somebody really should drop out and start a company, and I tell them not to, I’d be committing a moral crime. But most people probably should stay in school and actually get degrees, and it feels immoral to suggest otherwise. So, I felt trapped. I thought about it. And the absolute straight advice — 100% of the time, you should stay in school, finish your degree, not drop out. And I’ve concluded that because the people who are gonna drop out and start a company are gonna do it regardless of what I say, or what anybody else says. And so, by definition, it’s good advice. I can’t possibly steer anybody wrong.

      In general, actually, not only is it a good idea to get the degree. The thing that it’s the most underrated right now. I think the archetype/myth of the 22-year-old founder — it’s been blown completely out of proportion. The thing that is underestimated now in the Valley — and, frankly, Stanford is the ground zero of this — I think skill acquisition — literally, the acquisition of skills on how to do things — is just, like, dramatically underrated. People are overvaluing the value of just jumping in the deep end of the pool, because, like, the reality is, most people who jump at the deep end of the pool drown. Like, there’s a reason why there are so many stories about Mark Zuckerberg. It’s because there aren’t that many Mark Zuckerbergs. Like, most of them are still floating face down in the pool. And so, for most of us, it’s a good idea to get skills, you know, your degree or whatever, but then there is a lot to learn.

      If you want to, like, ultimately start a company, or go to a startup, there’s a lot to learn about how companies operate, right? There’s a lot to learn about how to deal with people. There’s a lot about how to manage. There’s a lot about, you know, leadership. There’s a lot about, by the way, finance. There’s a lot about legal. There’s a lot about marketing. There’s a lot about sales, HR. Like, there’s a whole skillset. Like, if you meet, you know, the really great CEOs, if you spend time with them — and you would find this to be true of Mark today, or of any of the great CEOs today or the past — like, they really are encyclopedic in their knowledge of how to run a company, and it’s just very hard to just, kind of, intuit all that in your early 20s. And so, I think the path that makes much more sense for most people is to spend 5 or 10 years getting skills. So, the problem with <inaudible>, it sounds great but, like, most startups are, like, really screwed up. Like I said, most of them just die in obscurity. And I don’t know exactly what you learn from dying in obscurity, but it’s not very much. A lot of people are at startups that don’t work well. They actually don’t carry away a lot of useful skills.

      Conversely, you know, you leave school, you go to a big company. A lot of what you learn in a big company is how to function at a big company, right? But the problem with people who have been at a big company too long is, in the cold light of day, when they go off to do their own thing, they literally don’t know how to function without all the infrastructure and support of a big company. And so, I think there’s a sweet spot, like a new high-growth company or the company that’s scaling. That’s probably the best place to go. And of course, you’re at Stanford, you have a huge advantage of being in the environment. You already know who those companies are, and, you know, you have a pretty good chance of getting jobs there. So, I think that’s generally really good advice.

      The other thing that I would say is, I have a favorite book I’ve never read, and actually, I’m worried about reading it because I think it can only disappoint me at this point, because I like the title so much. And the title of the book is “Smart People Should Make Things.” And like, as far as I’m concerned, like, that’s the entire value of the book. Like, I don’t even care what else he says. Like, just for engineers, it’s very obvious. Like, engineers should build things, should build products. And that could be open source, it could be, you know, working with a company or with a friend on something, but, like, going to a company that’s building something. But I think the same thing is true of everybody else, right, and people build all kinds of things. And by the way, the things that people build might be art, right. The things that people built might be, you know, businesses. The thing that people built might be an organization inside a company, or it might be a great explanation of something, but tangible output. I just always kind of really encourage people, like — when in doubt, fall back on building something tangible.

      Balaji: Yeah. And, like, we’ve got that thing at Andreessen Horowitz, right, like, works in practice, not in theory. So much stuff that I saw, you know, as a scientist, a Ph.D at Stanford, worked in theory but just not in practice, and there’s lots of stuff that’s just the converse, and only if you actually build it can you see it. Why did you and Ben, then, decide to start a VC fund rather than doing another startup?

      Evolution of venture capital

      Marc: Yeah. So, we were customers of venture capital, or at least I’ve thought about it that way. They thought they were giving us the money. I thought we were the customer. We had maybe occasional disagreements about that. And so we were customers of venture capital. I first raised venture capital in 1995, with my partner Jim Clark, from John Doerr — who was, actually, you know, an excellent VC for us at Netscape, and then we raised money from Benchmark in ’99 for Loudcloud, and that went really well. And then, between Ben and I, we also helped probably 100 friends of ours over the course of, sort of, a 15-year period, raise venture capital. You know, we were angel investors. We would help our friends go through it. And so you kind of view it, like, as almost going to the same department store every day for 15 years or something. After a while, you’re like, “You know, I think maybe I could do this, and I think maybe I have a few ideas from being on that side of the table.”

      So, we started really thinking about entering the business, and then we thought really hard about, you know — the traditional way to enter venture capital is to join an existing firm, because the history of venture capital is that the successful firms have all been around for 30 or 40 years, and we considered that. And then we basically got bit by the startup bug — me for the four-and-a-halfth time — and we decided that it was actually a good idea for a startup. We spent about a year and a half actually thinking about Andreessen Horowitz as a startup, and we spent a lot of time studying the models and talking to people who had been in the industry for a long time. And we ultimately resolved on what we thought could be two big differences. One was actually a little bit of a “Back to the Future” thing, which is — we decided that the general partners at Andreessen Horowitz would all be people who have been founders, or CEOs, or both, of tech startups. And, that kind of sounds like it might be obvious. Like, if you’re gonna have somebody on your board, and they’re gonna give you advice on what to do in your company, that maybe it would be helpful if they had actually done it before.

      It actually turns out, first of all — it had been a good idea in the ’60s and ’70s. The top VCs in the ’60s and ’70s, when venture capital was created had, for the most part, all been operators, and they had been legendary characters. Gene Kleiner had been famously one of the Fairchild, one of the original Fairchild people, one of the famous “traitorous eight,” who left Shockley to start Fairchild, left Fairchild to start Intel. Tom Perkins had actually been a general manager at Hewlett-Packard, which was actually, at the time, a source of a lot of the CEOs of the new companies in the Valley and actually, himself, had been a founder. He started a laser company, which was the kind of thing people did in the 1960s, and he actually raised venture capital himself and was a founder. Don Valentine. You guys had, I think, Mike Morris here last year. The founders of Sequoia Capital, Don Valentine and Pierre Lamond, both of whom are famous chip executives and entrepreneurs. And so, it actually was how venture capital got formed. Our analysis was basically, over the course of time, venture capital — a lot of the traditional venture capital firms had evolved where the successors to the founders were, in many cases, very successful investors, but were people who had not started and built companies themselves. And so, we kind of decided to bring that idea back.

      The other big idea that we had, that we’ve really pushed hard, is the idea of giving founders, and especially founders who have not been CEO before — we would use the term, sort of, give the founders superpowers — in the form of, basically, the world’s best network. And this is an observation that, you know, we’ve seen over the years. We’ve seen founders start companies, and then, at some point, the founder gets fired, and you bring in a professional CEO. One of the questions we always had is, what’s the catalyzing thing that causes the founder to get fired? And then what is a professional CEO? And professional CEOs, it’s always a type, right? It’s always like, you know, square shoulders, blue suit, six-foot-two, gray hair, fantastic teeth. Like, it’s a type. And what do these professional CEOs have that the founders didn’t have? And actually, some of it is, they have experience running a company, and we think we can help with that. But the other part is, they have these networks. They have been in the industry for 20 years, longer. They’ve got 20 years’ worth of, basically, network built up, right, and so they know customers, and they know other investors, and they know all the big tech companies. And if the company is to get sold, they know all the buyers, and they know all the reporters who cover the space, and they know all — if it’s a regulated business, they know all the government regulators. And so, they have these giant networks that they built.

      So, what we decided to do in our firm is, basically, essentially, pre-build the best possible network that any startup could have, and then basically let our founders plug into it, and basically get the superpower of having a giant network. The way that we did that is we actually have — we have a very kind of nontraditional structure. We have full-time professionals in our firm who are not general partners or investing partners, who are operating partners in six teams that build and run networks across categories, customers, investors, acquirers, executive talent, engineering talent, PR, and now, policy and regulatory affairs. So, we’ve got 85 people in the office every single day, and what they’re doing is they’re basically building and grooming a network on behalf of the firm, which then works on behalf of all the portfolio companies.

      Balaji: Andreessen-Horowitz is actually a network as a service.

      Marc: Yeah.

      Balaji: So then, one interesting point is — a16z was actually started in, you know, ’08, ’09, and it’s been, like, 7 years now, right? And the industry has changed, you know, the firm has changed, VC, more broadly, has changed. What are your thoughts on, kind of, that evolution?

      Marc: I would say there’s been more change — there’s no more change in venture capital in the last 7 years than probably in the preceding 20. And I’d also argue there’s probably been more change in the tech industry in the last 7 years than probably the preceding at least 15 or 20. There’s a bunch of new firms now that people are starting that are exciting. Another thing is seed investors — angel investors have always been important. Like, a big part of the history of the Valley is the willingness of people who have made, you know, some amount of money to write a check, and sort of fund the next idea. And you know, a lot of the original companies in the Valley, there was angel money involved. So, angels have always played a very critical role. In the last seven or eight years, it feels like a lot of the angels actually have professionalized, and when they do that, they renamed themselves — angel investors, to seed investors — because angel kind of implies an individual, whereas seed kind of represents a sort of investable asset class. And so a lot of the best angels have now actually raised funds, instead of just investing out of their own pocket, and they actually run these seed firms.

      And so, actually, we see kind of a restructuring happen in the industry where a lot of companies — companies used to just raise venture capital as their first round. They’d just go straight and raise a series A. And you could either raise a series A or you couldn’t. But it was only a very small percentage of founders who could raise an A round right out of the gate. You know, these days, it’s much more common to raise the seed round, you know, raise $500,000, or $1 million, or even $2 million as a seed round, and then go for a year or 2 or 3 — well, before you actually have to raise full venture capital. In fact, the seed phenomenon has now gotten so widespread that, now, the seed investors are trying to differentiate against each other. So now, there’s seed. There’s also pre-seed. There’s also seed extensions. There’s post-seed. There’s early A. And then, actually, below all of that, there’s incubator, accelerator kind of phenomenon. And so, we’ll actually sometimes meet companies that have raised, like, five rounds of seed capital in different forms. And so there’s just a lot more support in the infrastructure for a much larger number of new companies.

      I think that maps to what’s happened in the industry over the last seven or eight years, which I think is really remarkable — either we’re just taking it for granted or we haven’t really wrapped our heads around it. Which is, the history of the Valley for 50 years, from the 1960s through the mid-2000s — the Valley was kind of the best place in the world building, literally, computers — so chips, and then computers, and then software that runs on computers, but fundamentally building tools, right. Computers or software as tools. And then, you know, these giant companies, Oracle, and Sun, and Cisco, and so on, would build these great tools and then would sell them to customers. And the customer might be a consumer at home, but the customer, more often, was a big bank, right, or a big insurance company, or you know, a hotel chain, or somebody like that — or a car company.

      In the last seven or eight years, post the financial crisis, something has changed. Either the Valley is about to grow to become a lot bigger and more important than the Valley has ever been, or we are completely smoking crack. Many Valley companies still build technology and sell the technology as tools, but a lot of the best new Valley companies build technology and use it as a wedge to enter an end market, right? And so, as an example, the predecessor company to Uber was not, you know, a ride-sharing service that failed. The predecessor company was a little boutique software company that built dispatch software that got sold to taxicab operators, right? And there actually were companies that were in that business, it’s just, it was a tiny little business, because it turns out taxicab operators actually aren’t that excited about adopting new technology, they don’t buy very much IT, they don’t buy very much software. If they did buy software, they wouldn’t know what to do with it. And so, that was just never a very big business. And so, Uber and Lyft just come in and basically say, “Let’s just do it. Let’s just provide the ride. Let’s take complete responsibility for the customer service.”

      Elon Musk, of course, has pushed this to its logical conclusion, which is, you know, why not just build the car. I think that Elon gets tremendous credit, both for the car company and the rocket ship company, both of which are things that — nobody 10 years ago thought was possible to build either kind of thing as a new company, and it turns out that it is. It feels like the Valley is really expanding, basically — certainly expanding in ambition, and quite possibly, we believe expanding in capability, to be able to actually go directly into a lot of markets that historically you would have viewed as, you know, much more the province of existing banks, or existing car companies, or existing incumbents.

      Balaji: I think a big part of that is actually the fact that, if you’re selling IT to somebody, versus actually using it yourself, you can just recognize the benefits, you know, more obviously. Like, oh, if you’ve got your entire thing in a database, well, you can push out, like, a report of all ride times, and so on, and so forth. And they can understand and think about data, but the customer wouldn’t necessarily do that. It’s a major efficiency.

      Marc: If you’re selling technology to a company that’s then implementing it, it’s a layer of indirection. And there are companies — I mean, look, there’s, you know, Oracle got built to do this, and a lot of Oracle customers have gotten great results with Oracle. And salesforce.com just had a great quarter, and you know, they sell their stuff to lots of companies with big sales forces who do great with it. So, it works. But, yeah, we see this — we have this sort of, like — the term we use is full stack. Which is, you sort of see there’s a particular magic, exactly to Balaji’s point, there’s a magic that kicks in when you actually have complete responsibility for the end customer experience, and how the product or service is delivered. And then, especially these days, right, in the era of big data and machine learning, and all these things, there are things that you can do to optimize both experience, and then ultimately the economic model of the business. It’s become a very open question or a topic. Okay. So, how many industries are opening up where you could possibly do, you know, the equivalent of an Uber, Airbnb, or a Tesla, and these industries from the Valley?

      Balaji: I guess, let’s start taking questions, yeah.

      Audience Q&A

      Man 1: Hi. So, for a first-time founder who’s bootstrapping a v1 product, when do you think is the most appropriate time to first approach investors, and at what level? Is having a business plan and a team reasonable, a prototype to show potential, or demonstrable customer traction? Thank you.

      Marc: Yeah. So, it’s hard to give general advice because it really depends, but unquestionably, it’s better to have something working. Coming in with something working is a gigantic edge over coming in with nothing working, like, a huge edge. Even, by the way, for people who have done it before, people who have successfully run companies before, coming in with something working is a really big deal. And then it is, like, absolute magic. I mean, it’s like catnip to VCs if you can walk in and you’ve already got both the product and customers. Just rub it on us and it’ll drive us crazy. And this is another thing. Probably what’s overestimated right now is just raising lots of money — to be able to say you’ve raised lots of money. Probably, what’s underestimated is the bootstrapping process of getting in position with the core thing that you’re doing, and both the product itself and its value to customers, before you start raising a lot of money.

      Man 1: And with that customer traction and MVP all ready, like, what level angel seed A?

      Marc: If you’re a first-time founder, first-time founder, it’s almost always better to start with angels or with the early seed investors. It’s, again, contrary to myth and archetype. It’s very hard for the first-time founder to raise a straight A round. It’s almost always the case that they’re coming up through a seed. I mean, as an example, you know, Mark Zuckerberg raised literally angel money from Peter Thiel. That’s how he got started. He didn’t go and raise an A out of the gate. Sergey and Larry, same thing, they raised angel money. And so I think that that’s almost always the best thing for a first-time founder.

      Man 1: Thank you.

      Marc: Yeah.

      Man 2: You mentioned all the progress in AI, a new input, output, and all the language process. So, I have a very — if you have to pick, in 30 years, what’s the chance that we have a bot that does a better job in picking companies than Andreessen Horowitz?

      Marc: I hope to God we invest in it, because it’ll be the last investment we ever make. So, I mean, this idea is out there, right? And so, there are actually people literally trying to do this, and there’s actually a venture firm called Correlation Ventures that literally is trying to do this, or a version of this. And then, you know, there are people who are, like, data mining angel lists, and trying to figure out how to do this. And there are other people who are going about this. The computer scientist in me, the engineer in me, would like to believe this is possible, and I would like to be able to figure it out, and I’d frankly like us to figure it out. The thing I keep running up against — the cognitive dissonance in my head that I struggle with — is what I just see in practice — talk about in theory versus in practice. Like, in theory, you should be able to get the signals, like, you know, founder backgrounds, and this, and that, progress against goals, or whatever, customer satisfaction, you should be able to measure all these things. We just find, what we deal with every day is not numbers, right? There’s nothing to be quantified.

      What we deal with every day is idiosyncrasies of people. And under the pressure of a startup, like, idiosyncrasies of people get magnified out to, like, a thousand-fold. Like, people become, like, the most extreme version of themselves under the kind of pressure they get under in a startup, and then that’s either to the good or to the bad, or both. But people have their own issues, and then the interpersonal conflicts between people. So, the day job is so much dealing with people, that you’d have to have the AI bot that can, like, sit down and do founder therapy. Maybe.

      Balaji: Yeah. I mean, like…

      Marc: My guess would be we’re still a ways off.

      Balaji: Yeah. Like, just add to Marc’s point on that. I mean, the fundamental issue from, like, a machine learning standpoint is, you have very few events that are mostly returns, which are, like, these Facebook-like outcomes, right? And so it’s, like, almost like a rare event detector, like the Large Hadron Collider, right? You’ve got all these particles coming through, and you have to be able to predict, “Okay, which one of them is actually gonna make a lot of money?” That’s number one. Number two is, especially at the very earliest stages, you don’t have features in the traditional sense. Like, you don’t have a lot of really good data to work with, in terms of prediction. So, the later it gets, probably like series C or thereabout, you have enough, you know, systematic data to work with, but early on, it’s actually pretty challenging.

      Marc: Yeah.

      Man 3: Hi. Thank you. How are you guys thinking through your fund structure and the types of investments that you have to make as you raise more money? And can VC be, like, a winner-take-all market?

      Marc: There are a bunch of challenges to it. The central challenge is, any top-end venture capital firm that has a reputation that it wants to maintain, which is I think very important, can only invest in one company in a category. You can’t, practically speaking, invest in competitors. The company you’ve already invested in will feel it’s [a] betrayal if you invest in the new one, and then the new one will think, if you’re willing to invest in them, you must be very, like, dishonorable that you’re willing to betray your previous one. So, just — it doesn’t work. And so, like, the minimal number of venture capital firms has to be the number of firms required to fund the number of competitors, right, in each new market. And then we can debate — is that 3, or 5, or 20, or 40, or 100? And you know, certainly, we have too many venture capital firms. Like, we’ve got like 500 venture capital firms in the U.S., and certainly, there aren’t 500 competitors in every market, at least. There need to be at least a half dozen, dozen, you know, 15, you know, good firms to fund the competitors. We would love to make venture capital a winner-take-all.

      Man 4: I have a question with regards to blockchain and, like, the financial services industry. So it seems like there’s a lot of low-hanging fruit and a lot of far-fetched ideas that one could foresee using blockchain. So, I’m wondering, what advice would you give for someone who’s trying to see what is the best, I guess, niche area to target when you’re given such a wide array of potential use cases for the blockchain?

      Marc: Yeah. So, we actually shy away from giving advice like that. So, there’s two reasons for it. So, one is there is a concept called product-market fit right, which has become very fairly publicized now, you know, right product in the right market. There’s another concept we call founder-market fit, which is — is the founder of a company — is that the person who’s born to do that idea? And so, that question we tend to defer to the founders, because we figured the really great founders are gonna figure that — like, part of what makes a founder great is they’re gonna figure that out. The other thing we found is that it’s very hard — we have ideas for companies we’d like to fund, but we try not to talk about them too much, because we don’t want somebody — we don’t want founders to pick up somebody else’s idea. And it goes back to what Balaji said, which is, it is so hard to make a startup work. You have to be so irrationally committed to it. I mean, this is another thing. Like, startups are over-glorified in the sense of, like, people think they’re fun. Like, they’re not fun. Like, they’re not even remotely fun. Like, they’re punishing as hell.

      Balaji: I think it’s Bill Lee <inaudible>, it’s like chewing broken glass and staring into the abyss. That’s right.

      Marc: He said starting a company is like chewing broken glass. It’s, like, after a while, you start to like the taste of your own blood. Very vivid quote. But, like, it’s so hard, and it’s so hard because people are saying no to you all the time. It’s just no, no, no, no, constantly being told no. And you know, “Your idea is stupid, and, like, I would never do that, or why would anyone do that. This other company is gonna kick your butt.” And like, then your lead engineer quits. It’s just, like, endless. It’s got to be an idea that they feel so deeply about. It goes to, like, Balaji’s term, ideological mission. It’s got to be something where people feel so deeply that they have to do it, that they’re willing to tolerate that level of pain. And in our experience, most people aren’t willing to tolerate that level of pain for somebody else’s idea. And so, I respectfully decline to answer the question.

      Man 4: Okay, I see. No, it just seems like, for blockchain, there’s so many use cases, and for many of them, the timing could be completely off. Whereas, for example, for remittance payments, one could easily see how that’s a very easily applicable use case of blockchain, so.

      Marc: Yeah.

      Balaji: I’ll comment on this briefly. Basically, I think that remittances are to Bitcoin what VoIP was to the internet, in the sense of — it’ll work at some point. In the first 5 years or 10 years of the thing, it’s not high enough quality with the obvious alternative, namely VoIP versus landlines, or remittances versus legacy remittance systems to win. I think that, you know, Bitcoin, like Bitcoin as opposed to blockchain, but Bitcoin is good for transactions that are very large, very small, very fast, very international, or very automated. And you have to try to envision transactions that are, like, two, three, four, or even more of these kinds of things to think of things that cannot be done with the current system. If you think of things that cannot be done with the current system that are still useful, well, then, that’s 10x, right? So, that’s one way to think about it. 

      The other great thing about it is, like, Evan Williams’ thing, which is sort of vague, but it’s actually very useful. So, on the one hand, oh, a new technology, 10x, something that people haven’t done before. On the other hand, Evan Williams’ thing is, take a behavior that humans want to do and allow them to do it faster, better, cheaper, over and over. Take something that was once a rich man’s thing and make it accessible to the middle class, or take it from the middle class and make it accessible to everyone, right? And so, if you kind of combine those two things, the technology allows you to go, and in a way that was not possible. So I’d — you know, hunt in that general area. That might be something.

      Man 4: Thank you.

      Man 5: I co-founded two companies that faded into obscurity too quickly. You identified problems, and issues, and opportunities [that] it might take a startup, you know, weeks, if not months, if not years to identify. I’m kind of curious why Andreessen Horowitz and others don’t explicitly identify opportunities and problems, or even issue challenges or competitions. Then, so — I wanna delve in a little bit deeper — one of the things you’ve been talking about, Balaji, more specifically, is, like, the cloud versus the land and, you know, “software eating the world,” like, the divergence of the cloud. And I’m kind of wondering, in that world, where ownership seems to be more centralized, there could be some risk associated with that. I’m wondering if you could speculate about ownership in the future. I’d be interested, especially, talking from a blockchain perspective on asset management.

      Balaji: So kind of, there’s two separate questions there, and I think the first one is, why doesn’t VC pursue, like, an XPRIZE style model? That’s one. And then number two is, what happens with, like, the future of ownership, right? Kind of interrelated. So, the first one, I actually think would be a very interesting model for a fund. The reason I think that’s interesting is, one of the points Marc made is, and it’s one the most counterintuitive points about VC — no matter how innovative it is, an idea that comes across your doorstep today, there’ll be two more like it. My best example of that is Hyperloop, right? Like, so Hyperloop company comes across our doorstep, and like, a few weeks later, we have, like, two more that come in there. And so, what it means is that VC is all about filtering winner-take-all. So, the more that you can kind of push the tournament to inception, the more you can push the tournament earlier and earlier before you invest, the better. So, prize model, I think, could work. The problem is, of course, grading the prizes, judging the prizes, all of that type of stuff. That’s one.

      Number two, in terms of the future of ownership, I do think that, basically, the interface to every physical object will be ultimately digitized, in the sense that you won’t own a car, you won’t have — we already don’t have a book, you have Kindle, right, and you don’t have a house, you have an Airbnb, and so on, and so forth. And all of it becomes extremely mainstream. And what that means is that, actually, your mobility is vastly increased. And right now, we think of mobile as, “Oh, I can just go to Starbucks, and I can work from there, and it’s as much as I could work at home.” But I think, in the next 5, 10 years, it’s gonna be as easy to just jump up and move to another country as it is to just go down the street. What that means is the more internationally flexible you are — so, one of the big aspects of that, by the way, is the bank accounts. That impacts the blockchain aspect. One of the big things that’s a pain moving between countries — your Gmail works, your Facebook works, all your internet services work, those are IP-based, right — but all the nation-state-based things, like your bank account, are not quite as portable or as easy. And so, those kinds of things, I think it’s useful to identify all the prerequisites. So, as a thesis for, kind of, startups to look at, chop the things that anchor people to land, and I think you’ll have some interesting things there.

      Man 6: Hi. This question is for Balaji. I’m a freshman studying physics at the University of Illinois, and I was just wondering, what convinced you to continue on to do a Ph.D, and what were the skills that helped you on, like, in regards to entrepreneurship and whatever you’re doing today?

      Balaji: So, I would not do a Ph.D today. That’s my quick answer.

      Marc: So, why did you do a Ph.D.

      Balaji: Why did I do a Ph.D? Because I wanted freedom, in the sense of I wanted to do math and, you know, computer science, and so on, on my own time, right? But what I would have done instead is — I think the single most important metric for you guys to measure is your personal runway. In Silicon Valley, people think a lot about, you know, “Okay, how do I get an exit and get the money on top?” But they think much less about, “How do I minimize my personal burn?” So today, in the world, it is possible to just find a jurisdiction that is amenable to your preferences — that is warm, that is safe, that has good internet, and it’s really, really cheap. And so, you know, what I would do instead of getting a Ph.D, if I was just doing it today. First, I’d worked for a year at Google, or Facebook, or GitHub. I would have a job that permitted remote work. I would sacrifice the advancement to be able to work remote for the next three years or so on, and I would just save enormous amounts of money and live very, very cheaply. Every year that you work, you’ve got three years of runway. And so, that’s actually freedom. Once you have the ability to have, like, 3, 4, 10 years of runway, and you have the discipline of the grad student but the earnings of an engineer, right — so that’s what I would have done instead. So I wouldn’t do the Ph.D. I think you can learn and self-learn faster on the internet than you can, you know, in grad school. I think a bachelor’s degree is fine, like, you know. Like, I’m not saying drop out, or what have you, right now, at least. But I think you can do better than a Ph.D today.

      Marc: So I’ve got a question for Balaji.

      Balaji: Yeah.

      Marc: So Balaji is, for those of you who know him by reputation or know him — and tonight he’s done this — very big advocate for entrepreneurship outside the Valley, very big advocate for developing world entrepreneurship, very big advocate…

      Balaji: Why am I still here?

      Marc: …in this case, for, actually, literally, moving computers someplace else. I can’t help but point out that Balaji lives — where do you live?

      Balaji: Yeah, no. Unfortunately, I’m in San Francisco. But, but, but.

      Marc: Interesting. Interesting. Interesting. Literally, if you drew a circle around San Francisco, he’s right in the middle.

      Balaji: Let’s say that sometimes you have a goal that you have, it takes a while to get to because there are a bunch of prerequisites that have to be met.

      Marc: Right. He keeps saying he’s thinking about it. We’ll see. We’ll see.

      Balaji: I keep saying, no, no, I’m working on it. All right.

      Marc: He now is married, and he has a lovely little baby.

      Balaji: I do, I do. Those are all anchors.

      Marc: And the two of them are gonna have, I think they get votes.

      Balaji: They get votes.

      Marc: It’s my understanding of how this works. They get to contribute to the experience.

      Woman: Thanks, Marc. I’m from China. I work for Google China and [am] now a current student in the Stanford GSB. I’m really inspired by the entrepreneurship here, but I know there’s a lot of challenges for the immigrant entrepreneurs to start a company here. So, I’ve been wanting your advice for the immigrant entrepreneurs, especially for the first time.

      Marc: Right. I’m gonna turn that question over.

      Balaji: Sure. Okay. Yeah, sure.

      Marc: To the immigrant entrepreneur on the stage.

      Balaji: Yeah, yeah, sure. So, I thought about this a lot, and I’ve discussed this with Marc and Ben a lot. What comes after the dorm room entrepreneur is the developing world entrepreneur and the immigrant entrepreneur, but especially in the developing world. And I think, you know, one thing, you know, depending on what country one is coming from, and so on, obviously, there’s a wide range, but for someone coming out of India, for example, frequently making $100,000 is like making $1 million, in the sense of, like, the impact on quality of life, and so on, right? And there’s actually much lower-risk ways to make $100,000 than to do a startup, which is just extremely stressful, and you’re going for infinity, and so on, right? 

      And so I think that we’re gonna see new kinds of things, particularly as you get another billion, two billion people with cellphones, right? Like, then we’re gonna see new kinds of business models that are based on knowledge that folks outside of the U.S. and in the developing world have about their local economies, and also have maybe less than we have upside, more predictable returns, and they’re not quite as much of a, you know, roll-the-dice kind of thing. In some ways, if you start at zero, it’s easier to get to infinity, because you just have nothing to lose.

      Marc: Good, good, good. Thanks, everybody, for coming. Thank you. Thank you.

      Balaji: That’s good.

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Balaji Srinivasan

      Airspace as the Next Internet-Like Platform

      Sonal Chokshi, Eli Dourado, Grant Jordan, Jonathan Downey, and Samuel Hammond

      One of the most important lessons of the internet age is what happens when we give people — including companies, developers, engineers, hobbyists, and yes, even a few bad (or dumb) actors — a new platform, along with the freedom to innovate on top of it. For example, who could have predicted how profoundly the internet would change our economy, given how it started off as a research project — one where commercial applications were actually frowned upon in the early days?

      Now, the U.S. is on the cusp of opening up another such platform for commercial and social innovation: airspace (think drones, the non-military kind). There’s so many use cases for drones that we already know about, but what about new business use cases? And then, on the policy front, how do we calculate the risk of innovation on a platform made up of atoms (drones) vs. bits (the internet)? What are the pros and cons of registration? Because even though drones are like flying smartphones controlled by software, they’re also hard objects that could fall out of the sky … or go places where no one could go before, for better or worse.

      The guests on this episode of the a16z Podcast — continuing our D.C. and tech/innovation/policy theme — share their thoughts on safety, privacy, paper airplanes, and what they think are some of the most exciting things now possible in airspace. Joining the conversation are Washington, D.C.-based Mercatus Center tech policy lead Eli Dourado, along with graduate research fellow Samuel Hammond; Airware founder and CEO Jonathan Downey; and SkySafe CEO and co-founder Grant Jordan.

      Show Notes

      • The current state of drone technology and potential use cases [0:57]
      • Creative applications for using drones [10:04] and barriers to innovation [14:00]
      • Issues with safety [19:23] and privacy [28:55]
      • Discussion of new potential developments on the horizon [33:07]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal. And today’s topic, continuing our D.C. theme, is drones, and more broadly, policy and airspace as a platform for innovation. Joining us for that conversation, just to really quickly do the intros, we have Eli Dourado. He directs the technology policy program at George Mason University’s Mercatus Center. And he’s here with Samuel Hammond, who is a graduate research fellow in that program. And they’ve done a lot of research in policy reports around drones. And then we have Jonathan Downey, who is the founder and CEO of Airware, which provides operating systems for commercial drones, so enterprises can take advantage of aerial data for business applications. And we also have Grant Jordan, who is the CEO and co-founder of SkySafe, which is the company that provides security for airspace — for example, by taking control of and safely landing rogue drones. 

      And, by the way, those are both a16z companies, full disclosure. We have another drone’s investment, Skydio — which focuses on onboard intelligence by giving drones the same visual awareness and agility of human pilots — but they couldn’t join us today. So those are the intros. And now let’s just get started. 

      Innovation in drone technology

      I think the place to start off is, you know — Eli, you wrote an op-ed for me a few years ago, where we talked about airspace as the next platform for innovation. And I thought that was a really eye-opening concept for me, and I think we need to just break down what each of those terms mean. Like, what is airspace, why a platform, and why is it the next platform for innovation?

      Eli: Well, yeah, when I wrote that op-ed — I think it must’ve been about three years ago when I was first starting to get interested in drones. And the thing that I noticed immediately was that drones are completely legal to use for hobbyist purposes, and completely illegal to use for commercial purposes. And it reminded me of the internet back in the 1980s, right? The internet in the 1980s, it was this research program that the government had, but there were guidelines. You know, MIT had an AI lab, and the guidelines for students were, like, “You may not use this for any commercial purpose. We can get in trouble with the government.” It’s just illegal. 

      And I thought about the people running this program. They didn’t mean to, like, hold back the internet. They were, I’m sure, very well-intentioned. They wanted this to succeed, and they weren’t thinking about how much holding something back from commercial operation was likely to affect the success, right? If they knew where we were today, they would’ve allowed commercial use from the inception, right? And so to me, I was just thinking about — what would people do with airspace if they could use it for commercial purposes if it wasn’t — you know, we’ve had remote-controlled airplanes for decades. Hobbyists use them, so in that sense, consumer drones are nothing new. But you can’t do anything commercially. And so, what would people do if you could use it commercially?

      Sonal: And Jonathan, I think this is something you can weigh in on, because you are the founder of a company that is doing software for commercial drones.

      Jonathan: Yeah, happy to. And, you know, I totally agree. Three years ago, around the time you wrote that article, we looked at the global landscape for commercial drones. And most everyone at that time was thinking about consumer and hobbyist drones, and they were thinking about large military aircraft, but this idea of using drones for commercial applications was relatively new. And when we looked at the landscape, most of the companies who were doing anything meaningful in the space were international. They were in Australia, they were in France, they were in the UK. There were only a few countries where they either said, “Hey, we’re not going to regulate this at all. We’re going to allow it.” Or they said, “We’re going to have a process by which you can be a commercial operator of drones.” 

      And the UK and France were really leading the way there, with hundreds of commercial operations years ago. And many of the companies in the space said, “Well, we’re just going to ignore the U.S., or if we are in the U.S. we’re going to start all of our sales, and operations, and research and development outside of the country.” And only now with this Section 333 exemption process — and, hopefully, soon with Part 107 — do we see a lot of those companies that either started internationally expanding their operations to the U.S., or a lot more of these companies that are in the U.S. starting to get some commercial traction.

      Sonal: What is the Section… What was the regulation that you just cited?

      Eli: So, Part 107 is the new proposed small UAS rule and it will, sort of…

      Sonal: Buy UAS, you mean unmanned systems?

      Eli: Unmanned Aerial Systems.

      Sonal: Right, Unmanned Aerial Systems, yeah.

      Eli: So, this is the FAA term. But we call them…

      Sonal: Which, by the way…

      Eli: Cool people just call them drones.

      Sonal: Right, that includes drones, which just for a quick definitional thing on drones — because I think people actually are still confused by this sometimes. I noticed this in the early days of this community, an online community about drones — which is that drones are different than RC copters in that they can fly waypoints. And so they can follow, like, a preprogrammed path, and that you can — they’re not, like, remote-controlled, basically, in that context.

      Eli: So, the terms do get, like, blended, and so on. But yeah, I would agree that what we’re interested in here is some partial or full autonomy at some point. That you’re going to eventually be able to just tell it what you want it to do and it does it.

      Sonal: Okay, and so back to section whatever.

      Eli: So, the Part 107 rule is — back in 2012, Congress, actually with some foresight…

      Jonathan: It was actually with a lot of lobbying from large commercial industries. 

      Eli: A lot of lobbying, but good for Congress, they passed a provision of FAA reauthorization bill at the time to require that FAA come out with commercial drone rules by September 30th, 2015, which was, you know, several months ago.

      Jonathan: Came and went.

      Eli: It came and went. They’re still not out. But they’re coming. It’ll probably be sometime next month. There will be some permanent commercial drone rules. And in the meantime, as Jonathan said, we have these 333 exemptions that basically allows people to, sort of, negotiate or apply with the FAA to say, “I wanna operate commercially for this purpose.” And the FAA might allow it, or might not.

      Sonal: And what are some of the purposes and use cases that you guys are seeing people put to — I mean, what are the — why would they want those exemptions, basically?

      Jonathan: Yes, so the Section 333 exemption process was created last fall. And now just, you know, 9 months later or so, there’s over 4,000 granted exemptions in the U.S. for commercial use of drones across a variety of different industries. But we’re seeing a lot of exemptions in our own customers using drones in the insurance industry, in agriculture, in utilities, in all types of industrial inspections — oil and gas, land management, forestry, wildlife conservation. You know, that’s one of the things that I think is so interesting about this industry, is just the wide variety of applications and use cases. They’re really endless. We’re hearing about new ones all of the time. And I really think, you know, it makes that analogy to the early internet, you know, very, very real, in that it was designed and developed with several really important use cases in mind. But ultimately, when it was, kind of, released into the wild and, you know, not just people “with authority to develop it,” but really all kinds of people everywhere were adding capabilities to software and to the internet — is when we really saw this, you know, bloom in all the different uses for it.

      Grant: It’s totally the case that we don’t even know what the real killer apps for drones are yet. You know, I think there’s a couple of spaces that are, kind of, obvious that we’ve been thinking about so far of, you know — delivery, and inspection, and all these sorts of things. But there’s so much potential there, it’s, kind of crazy. You know, I think part of — a lot of our focus is on thinking a little bit further forward about the airspace management aspect. You know, I think using commercial drones early on in the process is pretty easy when they’re still very expensive. When the sorts of companies that can use them are still very limited. 

      You know, much like security in that early internet, right? When the internet is just a connection of a bunch of research centers and government, then it’s not — security is not really that big of a problem. Everybody on there is trusted. Everybody on there is doing the right thing. But as those barriers to entry start coming down — as, you know, anybody with 500 bucks can be flying a drone — suddenly it becomes, kind of, a different story. And suddenly, you know, people who don’t really know the rules of the road, who aren’t really sure what they’re doing, or have actual malicious intent — they, kind of, start coming to the party as well.

      Sonal: So you guys are, kind of, getting ahead of that. I want to take a step back for a moment and think about again — we’re all really reinforcing this concept of airspace as a platform for innovation. I think it’s actually, kind of, shocking to think about what it means to be able to do things in the sky. And I just want to take, like, a moment to pause on that.

      Eli: Yeah, and I think people don’t realize that, like, cell phone tower inspections is, like, one of the most dangerous jobs in America. OSHA called it…

      Grant: OSHA declared it the number one most dangerous job.

      Eli: …the most dangerous job. And that’s going to be…

      Sonal: Really?

      Eli: Yes.

      Sonal: Why?

      Jonathan: Yeah.

      Sonal: Just because it’s so high up?

      Jonathan: There were 14 deaths in 2013 alone from tower climbing.

      Eli: So, and there’s not that many…

      Sonal: Wow.

      Eli: …people who climb towers, right? So this is a fairly high percentage of the people who do this for a living. These are high towers and people fall, or something goes wrong.

      Jonathan: And that’s one that’s gotten a lot of publicity recently. There’s a lot of other jobs that are really dangerous as well. For two story steep rooftop inspections of residential properties, whether it’s during the underwriting phase, whether it’s during the claims phase — can also be a pretty dangerous job as well. And many companies — it’s, kind of, an opt-in job. They don’t just assign it to you, you have to, kind of, ask to be that person climbing up on the roof.

      Sonal: Oh, really? Because it’s that dangerous?

      Grant: It’s that dangerous, yeah.

      Sonal: Wow, so there’s clearly a lot of dangerous cases. What are some of the — I mean, oil and gas, that’s what I hear about all the time as an industry that needs drones. Like, why is that?

      Jonathan: Flare stack inspections for both onshore and offshore infrastructure in oil and gas. Also, oil derricks need to be inspected, and oil platforms need to be inspected for corrosion and damage on a regular basis. And a lot of these inspections, similarly, are very dangerous to do. Or, with the case of flair stacks, often the infrastructure needs to be shut down so that people can actually climb up on it. But with a drone in an aerial perspective, it gives you a completely different way to assess the status of the infrastructure without having to shut down critical equipment.

      Practical uses for drones

      Sonal: So we’ve, kind of, outlined some of the more dangerous jobs and use cases that drones can help address. Now let’s think about some of the things that there are opportunities you wouldn’t have had if it weren’t for drones. Because when we talk about all those application use cases — insurance, agriculture, etc. — in a lot of ways, we’re talking about disintermediating existing alternate approaches that are expensive, or prohibitive, or difficult, or unsafe. Like, having to do ladders, or oil and gas inspection. Things that are just impossible. I think it’s especially interesting on the creative side, like, what you can do with photography, aerial Hollywood filmmaking, and some of the really creative aspects of this. Because to me, the internet wasn’t just a commercial platform, it was a creativity platform. And I’m curious to hear what you guys are seeing on that front as well.

      Samuel: I’d just say, first of all, that that qualitative part gets missed in, like, FAA cost-benefit analyses, you know? So, how do you put a price on a vista that you haven’t seen before, right?

      Sonal: Exactly. Especially when you don’t know what it’s going to look like, because that’s the whole point of — again, not to be platitudish, but that’s the whole point of innovation. Like, it’s supposed to surprise you in terms of what’s possible. We can list all the use cases we want in this room, but we truly have no concept until we see companies and people start really inventing things around it.

      Grant: Well, yeah. I was actually going to say on the creative side, one of the things that I think is some of the most interesting stuff is on filming — just being able to do shots that used to require helicopters, that used to require, you know, tremendous amounts of coordination, and time, and money. And now it’s just, like, you know, a drone just allows that so easily. So, kind of, the entry-level for what used to be a helicopter shot is, like, nothing now.

      Jonathan: Well, and whether it’s in Hollywood and taking shots that were previously from manned helicopters, or whether it’s the utilities industry, you know, getting high-resolution photos of power lines from what used to be manned helicopters — that’s the starting point. But then it gets really interesting when you start understanding what wasn’t even possible with manned helicopters, and now is becoming possible with small aircraft — including, you know, new shots in Hollywood that previously — you know, you can’t get a helicopter within 5 and 10 feet of a person and, kind of, circle that person. But you can do that with a drone, especially as these things are becoming smaller, safer, lighter-weight.

      Grant: Right, or even shots that transition from inside to outside.

      Samuel: Passing through windows and so on. I’m no Hollywood expert, but I imagine in the past that they’d do a shot that pulls into the window, and then there’d be some hidden cut, and then reset the shot from inside. Now you could presumably just open the window.

      Jonathan: The corollary for a lot of these industrial inspections is the ability to do things like fly underneath a bridge, fly underneath an oil derrick. The top side of some of this infrastructure was always accessible with manned helicopters, albeit at a very expensive price tag. But now you can go inside of buildings, inside of — I mean, we’re seeing companies do inspections of the insides of everything from, you know, large oil containers to even large gas turbines.

      Eli: What’s interesting to me about the Hollywood application is that they actually were some of the earliest adopters, and they adopted it even before it was legal for them to do so. So Hollywood is, like, the…

      Sonal: Follows its own rules, goddamnit.

      Eli: Hollywood was, like, the Uber of drones and, sort of, like, they were just going to do it and ask for forgiveness and not permission. So, that was one thing I think, yeah, Hollywood helped with the drone policy.

      Grant: Yeah, well, and also Hollywood actually helped a lot on the 333 exemptions. Right, because they, kind of, had some of the biggest immediate incentive to get commercial use approved. And also, what I thought was, kind of, cool is, you know, they paved the way for the use of drones in filming, because they already had all of the safety procedures, and flight manuals, and things that they had been using for manned helicopter shoots for years. So, they literally just took those and shifted them over and, you know, called it, “This is the manual for shooting with drones.” And that’s what, kind of, lets you get the 333 exemptions for shooting so easily now.

      Samuel: So, we discussed some of the innovation arbitrage around drones. When it comes to Canada, some of that was also film, because there’s film industries in Toronto and Vancouver. And they could make shots and not have to worry about asking for forgiveness.

      Sonal: So it’s interesting you reference innovation arbitrage, because the example you shared is basically — and this goes to what Jonathan was saying earlier about some of the innovation happening around the world — is that when certain places have more regulatory flexibility, it then draws that industry correspondingly. And I think the reason this causes U.S. lawmakers to freak out a little bit when they hear, you know, Amazon saying, “Hey, we might start developing drones in Brazil or another country,” it actually becomes correlated to a direct loss of the economic opportunities that are provided as a result of this. 

      I mean, I’m thinking of the internet example. It’d literally be like saying, “You know what? We’re not gonna let commercial applications happen on the internet. So let’s develop them in China, and India, and Brazil, and South Africa.” Yeah, I’m just listing the bricks right there. But that is, kind of, the risk, to me, at stake here when we talk about this. Because I think, again, people are really underestimating how much is possible in the air. I don’t mean to be cheesy, but drones excite the fuck out of me because — and no, they really do — because it’s insane to me that, you know, we talk about men, women, humankind wanting to, like, fly. And now we’re talking about a whole new level of excitement, to be able to reach into the air and do things. I mean, don’t people get that, like, this is a really big deal for God’s sake?

      Samuel: Adam Thierer from Mercatus Institute and I have a paper forthcoming called Global Innovation Arbitrage, and drones are a major case study that we look at.

      Sonal: Why’d you guys pick drones?

      Samuel: Well, we picked, you know, drones, we picked genetics. So, there’s these big things where, first of all, they’re major emerging tech, and they’re very much in the news. I picked drones personally, because, as a Canadian, seeing drones in the industry take off in my home country. And so, we actually focused on Canada and Switzerland. And the Swiss government has taken a very risk-based approach to drone regulation.

      Sonal: So, is that a bad thing or a good thing?

      Samuel: A good thing.

      Sonal: What do you mean by risk-based?

      Samuel: A good thing.

      Eli: They’re actually evaluating the risks and moving forward accordingly, as opposed to just blanket bans or…

      Samuel: And it’s flexible for that reason. So there are very few — for example, in Switzerland, there are very few bright-line restrictions on what you can or cannot do. They’ll have guidelines about, for example, going beyond line of sight. But those aren’t written in stone, such that if new technology makes that safer, that they can’t be revised, sort of, on the spot.

      Sonal: Right, because the current law, if I’m not mistaken — at least locally, I know that you do have to keep drones within your visual line of sight. And that, sort of, seems to defeat the purpose, that the very purpose in certain use cases, like, if you’re a farmer mapping your fields is to be able to go beyond the line of sight.

      Samuel: So when I reached out to the Swiss government on this — and this is another analogy of the early internet. I was really asking them for an estimate of how much commercial operation is going on in their country. And the reply I got back was, “We’re not the United States. We don’t keep a tab on every single commercial entity.” So…

      Sonal: That’s fascinating.

      Samuel: And, sort of, like, the early internet, you could look up every website in, like, a phone book, right? That’s, sort of, the mentality that still exists in the U.S. with drones and the Swiss and the Canadians. They’re comfortable not knowing exactly how many commercial operators there are.

      Sonal: Right, and in fact, isn’t part of the problem — I mean, in the current state — that the regulation process can be prohibitive? I mean, the registration could be really expensive for small operators. I don’t know enough about it — like, what’s…

      Eli: Well, there is a consumer registry now. And that registry, it’s not expensive — it’s five dollars — but it could potentially stop people from taking that step. There’s, I think, a lot of lawbreaking going on right now, because I think there’s something, like, a million-plus consumer drones, and only 400,000 have registered.

      Sonal: Are registered, right?

      Eli: So, we’ve created a new law that has turned us all into a nation of lawbreakers. But then the other thing is this isn’t — the registry is not very well-tailored to the actual risks that we face. You have to register a drone if it’s more than 250 grams, which the FAA helpfully put that…

      Sonal: Which, by the way, how much is that in pounds?

      Grant: Well, the FAA…

      Samuel: 0.55.

      Eli: This is two sticks of butter, is what the FAA says.

      Sonal: Oh, really?

      Eli: That’s the way, yes.

      Sonal: Oh my God.

      Eli: So two sticks of butter or bigger you have to — so it’s, like, half a pound, well, 0.55 pounds.

      Sonal: It’s insane.

      Eli: If it’s that big, you have to register it, right? And one of the things that we’re looking at is how dangerous is it, you know, to have a more flexible standard that goes up to, like, 2 kilograms, which is what’s used in a lot of other countries.

      Sonal: How much of that in pounds, 2 kilograms?

      Eli: 4.4 pounds.

      Sonal: And so this is not even — this is just the weight of the drones themselves.

      Eli: Of the drones, yeah.

      Safety concerns

      Sonal: It’s not including any, kind of, payload, like, for commercial applications when you’re doing delivery. <crosstalk> It counts the payload, okay. So, let’s take a step back for a moment and just talk about the safety implications of drones. Because what’s different, obviously, with the internet, and drones, and any airspace objects is that they are flying — we call them flying smartphones or flying computers, we think of them that way. But they are flying objects that can fall out of the sky, like, hit your head, they can get in your tree, they could kill your cat. I mean, I don’t mean to be frivolous about it, but these are realities. So, let’s talk about the safety implications of drones. And what are some of the concerns that people have, and that you guys who are really involved in this space have heard?

      Eli: Well, I think there’s two things that people are worried about, and one is, as you say, falling out of the sky, hitting people on the head. And to me, that is something that we can properly deal with through the tort system. Just in the same way — if you hit somebody with your car, like, they sue you. Maybe the insurance company…

      Jonathan: It’s a great example. It’s an example, though, that requires liability insurance to drive on public roads, and that requires registration of your car. So, I might be the odd person out here, but in the same way that you more or less have to have a MAC address to get on the internet, there should be some mechanisms by which we identify the other people who are flying drones near us, or flying drones. I was having dinner about a year ago, actually, in Berkeley, and a drone flew right into the side of the restaurant, crashed, and then just about fell on top of the head of this girl who was standing there. So, you know, I think it is an…

      Eli: There’s a lot of people out there doing dumb things with drones.

      Jonathan: There are some people doing dumb things. And I think we can keep the registration requirements and things like this very, very basic, and easy…

      Sonal: Lightweight, yeah.

      Jonathan: …and lightweight, but structured in a way where, you know, the person has an incentive. If there’s some identifying marking on that drone that’s going to say whose drone it is, people are going to be incentivized and maybe think twice before they, you know, fly their drone into the side of the building or, you know, above New York City and into a skyscraper.

      Sonal: So, you’re saying associate some, sort of, identity or location.

      Jonathan: That’s the idea behind registration, is that if you’re flying this drone, especially in a public space, and something goes wrong, people will be able to identify whose drone it is.

      Eli: But then is it really 250 grams that’s the right threshold for that? I mean, I’m not worried about a 250-gram drone.

      Sonal: Two sticks of butter falling on your head, I hate to tell you, nothing’s gonna happen.

      Eli: Two sticks of butter falling on my head, I’m not too worried about that, personally.

      Jonathan: I think that that 250-gram allowance could be called the paper airplane allowance.

      Eli: Right. Well, because actually under federal statute, paper airplanes are aircraft.

      Sonal: Wait, are you serious?

      Eli: Yes, yes.

      Jonathan: Yes.

      Eli: So paper…

      Sonal: What?

      Eli: …airplanes are… The FAA…

      Sonal: So every time a kid, like, folds up a paper airplane in the classroom in kindergarten, they’re, like, breaking the law? Or they’re not registered?

      Eli: The FAA is forbearing on enforcing standards…

      Samuel: Sort of discretion.

      Eli: …the registration standards on paper airplanes.

      Sonal: Oh my God.

      Eli: So, the FAA would say that they have the right to regulate that. I mean, they would be embarrassed to say it, but they would say that they do.

      Sonal: So, Grant, when you guys, you know, started thinking about this, like, thinking far ahead about, like, okay, the same way the internet needed, like, security — and airspace will need security, in this way where you can essentially enforce, so to speak, the anti-drone —what were the scenarios that were coming in you guys’ minds that you came up with this?

      Grant: Yeah, well, I mean, it’s kind of interesting, too, because when we first started working on this and thinking about this, you know, it was still pretty early in the space, and we weren’t really seeing drone incidents occurring. You know, whereas now it’s, like, literally one a week, if not more. But I think to me, the big difference is that there’s, kind of, a gap in airspace enforcement, right? Like, if you’re talking about commercial aviation, civil aviation — at some point your enforcement of airspace restrictions — there’s, kind of, two things that come into play. One, is just the barrier to entry to be involved in aviation at all, right? You know, the amount of training required, the amount of money required upfront with planes, and fuel, and all of that. But then, you know, in addition to that, it’s a question of, at the end of the day, there is an enforcement mechanism up there. You have your plane registered with who owns it. The FAA can come and just cite you, can take away your licenses, things like that. You know, they can track you down. They have transponder requirements. And also, you know, at some point, if you fly into restricted airspace, the National Guard will literally, you know, fly an F-16 up next to you and tell you to land.

      Sonal: They shoot you down like, “Top Gun.” Sorry, I don’t mean to get all dramatic. It’s just…

      Grant: Yeah, but, you know, the F-16 and the National Guard doesn’t really help when you have, you know, just a quadcopter flying somewhere, that’s not really an appropriate level of response. But, you know, the spread of drones on the consumer side really, kind of, brings in this level of — it, kind of, changes the rules in thinking about airspace security, facility security, things like that. You know, when we talk about how it redefines various ways that we think about things, you know, if you’re thinking about perimeter security of something like a power plant, like, a nuclear power plant or something — or a prison, for example — there’s a lot of assumptions we make in building a security perimeter about fences, right? But now that you have drones, you know, an 8-foot fence versus a 20-foot fence versus a 2-foot fence is pretty much equivalent. It doesn’t really matter. You know, you can fly your drone over and deliver your contraband, kind of, regardless of the height of the fence. So it, kind of, just breaks down a lot of our, like, traditional security models.

      Sonal: And so you’re thinking about it more in the sense of how to — because I know there’s geofencing, where you can actually, like, fence in a region that a drone is, sort of, contained to fly, but you’re talking about when you can’t control the perimeter, so to speak, in military parlance.

      Grant: Yeah. I mean, you know, the problem at the end of the day with something, like, geofences, right —which, obviously, totally necessary, totally a step in the right direction. But you’re trusting the device…

      Sonal: The operator, right?

      Grant: …you know, to essentially police itself. You know, you’re saying, “Drone, don’t fly here.” And it’s going to abide by your rules. And that’s really good for eliminating, kind of, the initial low-hanging fruit of the people that are going to follow the rules. But, you know, a really good case in point is literally anyone flying a drone right now within a 30-mile radius of Washington D.C., right? Right now, in order to do that, you need to essentially override those controls, because that 30-mile radius is a no-drone zone — which also, incidentally, is, kind of, confusing if you are an RC hobbyist who’s been living near D.C. for decades flying RC planes, but now because it’s a quad rotor instead of a traditional RC plane, now it’s not okay to fly there. It’s, kind of, a confusing set of rules.

      Sonal: Right, so you’re thinking about the enforcement aspect. Well, I want to think about the other safety issue, or is that the — you were mentioning there’s another one.

      Eli: So, the other one I was thinking of was collisions with planes in the air. It’s the one that — the manned planes. So, this is something Sam and I have done some research on, and we use as a, sort of, parallel phenomenon — planes hitting birds. So, there’s actually many orders of magnitudes more birds in the airspace than there are drones. And so, what sort of conclusions can we have? And the FAA actually has 25 years of data on, sort of, voluntarily-reported bird strikes. And so we looked through that.

      Sonal: What’d you guys find?

      Eli: Birds are pretty safe.

      Sonal: Okay. Thank God, I love birds.

      Eli: So, we do hit birds all the time. Sometimes they cause damage, sometimes they do cause injury or fatalities. But in the context of — just the massive, massive number of birds and all of the flights — manned flights that we have, it’s actually a very low rate at which they cause any threat to humans.

      Sonal: And how does this play out with drones?

      Eli: So what we think is that the evidence seems to show that for small, you know — one of the things we do is we look at individual drones versus swarms of drones, because birds, they fly in flocks. So we look at the subset of bird strikes where it’s just a single bird. And then we look at the species of the bird and assign it a weight based on the average mass of the species. And so, what we found is — we looked at the 2-kilogram thresholds, and so that’s what’s used in a number of countries.

      Sonal: More than a stick of butter. Two sticks of butter.

      Eli: More than a stick of butter. And that’s what’s used in a number of countries for the threshold for what can be unregulated. And for a 2-kilogram drone, I think we found that there might be a human injury once every 187 million years — continuous flight hours…

      Jonathan: Continuous flight hours…

      Sonal: Wow.

      Jonathan: …of the drone.

      Eli: …of the drone. And then if you look at commercial jets, it’s even smaller. I mean, I think I got — I think the last number that we got was 41 billion years of continuous operation. Which is, you know, 3 times the age of the universe. So pretty safe. I’m not very worried about, like, a 2-kilogram drone taking out a 737, or anything like that.

      Sonal: Right, well, I think one of the funniest videos I’ve seen on the internet, and I’m sure you guys have all seen it, it’s this one that — it goes every viral every now and then, and there’s always a different version of it. Like, of an eagle battling a drone.

      Eli: Yeah.

      Jonathan: Eagle versus drone, kangaroo versus drone.

      Eli: Yeah, spoiler alert.

      Sonal: Oh, was it…what were some of the other animals?

      Jonathan: Kangaroo wins.

      Eli: Yeah, yeah, it’s a kangaroo.

      Sonal: That’s great.

      Eli: Yeah, I mean, we’ve learned about birds that they’re more territorial than a lot of us knew. I guess scientists knew this, but the rest of us didn’t realize how territorial birds are.

      Privacy issues

      Sonal: Right. Well, then the third category of safety that I think has come top of mind for a lot of people is privacy. So, one example this weekend I thought was really interesting is — the New York Times decided to fly and use drones to see the mass graves that were being unearthed, and they weren’t allowed to look at them. And I thought it was great. I saw that — oh, I think it was someone on Twitter, Jenna Wortham or someone said, “Hey, we sent in drones.” And I’m, like, “Yeah, that’s great.” And then I was thinking of the counterexample about it — an individual level, where you might have a star who wants her privacy, and she doesn’t need to be spied on by paparazzi. And they’re going to use drones, just like anything else. And so, what are some of your thoughts on that use case, and that concern?

      Jonathan: I think this is a case where, you know, technology can really be a significant enabler. You know, 10, 15 years ago, you know, if you had asked everyone whether they were willing to carry around what’s essentially a GPS tracking device in their pocket, people would’ve thrown their arms in the air and said, “Absolutely not.”

      Sonal: Totally.

      Grant: But with a couple, you know, technology additions to your cell phone, and the allowing you to turn the GPS on and off, and delegate which programs have access to it, and when they have access to it, and opt in to allow 911 to have access to your GPS position.

      Sonal: You feel safer.

      Jonathan: People are quite comfortable. They actually are happy to have that on them. And now there’s a variety of different applications around, you know, enabling people to run by themselves and alert someone if their GPS position ever stops for five minutes or more.

      Sonal: Parents use it to track their kids for safety.

      Jonathan: Parents are using it to track their kids. So, all of that, I intend to just be, kind of, an example of — I think the same thing is playing out here with drone use. Which is to say, yes, this is a technology that could be used to invade people’s privacy. But with some basically, you know, technology controls on it, it can also add a tremendous amount of value to society, without invading people’s privacy. And so there are technology mechanisms to — if you have permission to fly over Property A, and Property A abuts Property B, it’s relatively easy to make sure that photos that are taken over Property B are immediately deleted, or are never taken at all. Or, photography is only taken over the property that you have permission to fly over.

      Sonal: You can do a lot through technology.

      Jonathan: You can, and you can have, you know, geofences that allow for you to only fly within the bounds of properties that you have permissions to fly, or Section 333 exemptions to fly over, with the permission of the property owner.

      Sonal: Right, and besides technology, there [are] also existing laws that cover so much of this, like, the Peeping Tom case. Like, why do we need new law, when there’s already…

      Eli: No, that’s right. I’ve heard policymakers say, “Well, what if this, like, goes up right next to my bathroom window and looks in?” And the answer is, there’s already a law against this. It’s probably a state or local law, and it would just be enforced in exactly the same way. So, there doesn’t need to be, I think, a drone-specific…

      Sonal: Preemptive.

      Eli: …rule for that.

      Jonathan: And these types of laws should be technology-agnostic. It shouldn’t matter whether it’s binoculars used, or whether it’s a drone used, or a ladder.

      Eli: That’s right.

      Sonal: I like that idea — that it should be technology-agnostic.

      Samuel: But in terms of how the law should evolve, as long as we’re treating drones as any other kind of airplane — I wouldn’t recommend this, but if a drone is trespassing over your property and you choose to shoot it down, it’s, like, treated as shooting down an airplane.

      Eli: Shooting an airplane, which is, like, 25 years in jail, or something like that. I don’t actually know the penalty. But this is — so, don’t shoot down a drone right now. Don’t be the test case.

      Sonal: Right.

      Grant: Well, I guess, yeah. As far as [a] test case, I mean, that’s part of the question here — is that regulatory-wise, we don’t really know where we stand on that piece, right? You know, so far we have one piece of case law of — in Kentucky, apparently, you can shoot down a drone with a shotgun and you’re fine.

      Eli: Well, the FAA I think has issued — they’ve preempted that, and they say, “This is a federal offense, and it is shooting down an airplane. And it’s the same.”

      Jonathan: Yeah, it’s pretty clear, both from all of the, you know, past laws that exist and then right now, there’s language just to reinforce it in FAA Reauthorization Act of 2016 — just further clarifying the Federal Government and the FAA’s ability to preempt all state and local laws as it pertains to the national airspace.

      New developments on the horizon

      Sonal: Great. I’m just surprised it wasn’t Texas and it was Kentucky. So, what excites you guys? I mean, we’re talking about some of the — just to switch gears from the safety topic again, and go back to, like, what’s new and exciting. So talk to me about what’s interesting to you guys. You guys are on the forefront of watching the trends in this space. I want to hear what’s new and interesting.

      Jonathan: I guess from our perspective, right now in the United States, we’re still at this stage where the military has been using drones. They’re using them every single day. You have millions of consumers, literally, who are using drones in many cases every single day. And the major commercial companies have yet to really step into the fray and move from testing of the technology, which is where most all of them are at today, to actually using it as the way they do things. For whether it’s, you know, the insurance industry underwriting claims, catastrophe response, or utility inspections, replacing climbing up towers, and manual flights with helicopters with drones. So, I think that’s the thing that’s most exciting to me, and something that I also expect we’re gonna see in the next 18 months — the commercial companies actually move forward with, you know, commercial drones and aerial data as a way of doing business.

      Grant: I’m super excited about airspace integration, and about us getting to the point where we can actually have, you know, large quantities of commercial drones in the airspace, you know, kind of, interacting with commercial traffic. Routing correctly, and things like that. You know, and it’s the kind of thing where it’s going to take a concerted effort by a lot of different groups coming together. You know, you can’t have Amazon using drones, plotting their own paths, and Google using their drones and plotting their own independent paths, with no interchange of information between them. 

      Like, the NASA UTM program is working on a lot of that stuff. And I’m super excited about that. You know, super excited about transponders on aircraft, you know, things being registered properly — you know, actually having accountability. And once we get to that point, where we’ve got integration, where we’ve got accountability — then that just, like, opens up the door to all these different uses. And being able to have a point where a company, you know, can sit down and say, “How can we use drones? Okay, this is a thing that would help us.” And then there’s just [an] unknown path to actually use them correctly. You know, right now we just have so much gray area left in that system. But once we get that cleared out, I think it’s going to be great.

      Samuel: Yeah, I’m excited about the entertainment side. So, I follow some of the hobbyist goings-on around first-person view drone racing. And these are people who put on goggles and fly drones around tracks. And I think it’s just — I think we’re only a year or two out before this is broadcast on ESPN, because just — it’s some of the most exciting things to watch.

      Grant: Oh my God, yeah.

      Samuel: Oh, yeah.

      Grant: Just, the few groups out there that have started doing drone racing and have really, kind of, tried to address how to make watching that exciting — they’ve done an amazing job. Like, watching Drone Racing League and a few of the others, like — it’s going to be super sweet. I also, I’ve been a little disappointed in drone racing, just because now that I’ve started watching these guys that are doing it now — these guys and girls — they’re already so good. I’ve already had to, like…

      Sonal: You can’t keep up, Grant.

      Grant: I can’t keep up. I’ve already dashed my dreams of becoming a professional drone racer, because clearly I’m not good enough even for this early stage of the sport.

      Eli: And the other thing, going along with the drone racing, which is all first-person view — I mean, just imagine putting on, like, an Oculus Rift headset, and just taking a drone up, and looking out of the drone’s camera and just being…

      Sonal: And being in the sky.

      Eli: So, you get to, like — it’s, like, being a warg on “Game Of Thrones,” right? It’s, like, you get to experience flight as if you were a bird or, you know, I guess as if you were a drone. I think it’ll be, like, really fun.

      Sonal: I love that. It’s like, the e-sports version, and now we have, like, drone sports — kind of, like, a different version of, like, digital sports, essentially.

      Samuel: Right, and I’d love to participate, and I’d love to get in on that, but I live too close to the White House.

      Eli: And then I think, pushing it forward a few years, why do these all have to be unmanned systems, right? Why can’t you have an autonomously piloted aircraft that carries a human, right? Why shouldn’t we get human pilots out of the cockpits, just as we’re getting them out of, you know, away from behind the steering wheel in cars? You know, you can imagine a world where robots are us flying around, and that’s way cheaper, and you don’t have to carry a pilot. You don’t have to pay a pilot. Maybe we could have air taxi systems that are economical again. Maybe just huge safety benefits. I think in general, aviation — something like three-quarters of all accidents are pilot error. In commercial aviation, I think it’s about half. And so, you know, safer. What can we do to redesign airspace? The FAA is working on [the] NextGen airspace system, where there is more machine-to-machine communication. And so, you can have better routing. Well, what does that look like when that gets adopted? And how does that improve?

      Sonal: That was actually one of the interesting quick sidebars about Jonathan’s notion of identity, to urge some kind of registration tied to, like, an entity — what was interesting to me — what I immediately thought of — just, like, IP address and the internet. Like, you essentially can have all these drone nodes communicate with each other and route information as a result of that.

      Eli: Yeah, and some of the plans are actually very similar to how the internet is structured, in terms of public key infrastructure and, sort of, using…

      Sonal: Right, exactly, pack it.

      Eli: …like, SSL certificates.

      Sonal: Right, exactly. It’s super fascinating.

      Jonathan: There’s just no need to reinvent all of this technology just because it’s being used with drones.

      Eli: Yeah. And I would say the last thing that I’m excited about that’s in aviation generally, but it’s perhaps somewhat unrelated, is supersonic. Because we haven’t — we’ve had a complete ban on supersonic in the United States, over land anyway, since 1973. We haven’t had a commercial supersonic jet since the Concorde.

      Sonal: And what does supersonic do for us besides make a loud boom?

      Eli: You could go cross-country in two hours, right? So I could come from DC, fly in, record a podcast with you, and then fly home. Like, that day.

      Sonal: It’s like the hyperloop of the sky.

      Eli: Yeah, hyperloop of the sky.

      Sonal: I love that. I’m, kind of, excited by the art aspects. And when I think about this in the context — when people think of swarming drones — I love drone swarms. I think it’s amazing to see this orchestration of multiple drones in the air. And people view it as a very menacing thing, but I think there’s something very artistic, and elegant, and beautiful about it. But the other thing that really excites me — when I think of movies. Like, you know, when they redid the first three “Star Wars” movies — which were just awful, for the record, as everyone probably in this room agrees — I love the visual, though, of the fact that you had all these aircraft in the sky, and that people could jump from one aircraft to another. And even in this movie, “The Fifth Element,” which is this really lame, fun movie, there’s this amazing scene of people literally doing the same kind of thing. Like, they’re in the air and there’s layers — not just, like, one layer, but there’s layers of aircraft in the air. And it just gives a sense of actually living your life in the clouds, you know, where you can actually have, like, cafés in the air. You can do things in the air. I know that sounds a little crazy, but to me, when I think of airspace, I just think it’s amazing to me that we can now build upwards in ways that we couldn’t before. Well, thank you, guys, for joining the “a16z Podcast.”

      Jonathan: Thank you.

      Eli: Thanks to you.

      Grant: Thanks for having us.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Eli Dourado

      • Grant Jordan

      • Jonathan Downey

      • Samuel Hammond

      How to Be Original and Make Big Ideas Happen

      Adam Grant and Sonal Chokshi

      From Aaron Sorkin to Steve Jobs to Meredith Perry and Elon Musk, “original” thinkers — such as entrepreneurs — do a lot of different things to move the world to their visions. And many of those things (and traits) are counterintuitive, such as … Embracing procrastination. But there’s a catch: It’s about being the just-right amount of procrastinator, expert, or confidant. There’s a curvilinear relationship between too much and too little.

      There’s also some surprising findings about why NOT to “start with the why” but with the how. Because sometimes the how is much more believable than the why. Especially when it comes to getting people to engineer things from ubeam to SpaceX. Or to really being able to tell the difference between communication vs. confidence vs. competence.

      Ultimately, it’s all about being flexible, argues top Wharton management professor and New York Times columnist Adam Grant in his new book Originals. So how do we strike the just-right balance — whether making an entrepreneur or just trying to raise more creative, productive kids? Is the answer perhaps to immerse them in sci-fi books and video games? Well, J.K. Rowling could be the most influential “original” alive, argues Grant in this podcast… but not for the reasons you think.

      Show Notes

      • Discussion of what a non-conformist is, and different types of procrastination [0:00]
      • Steve Jobs, Meredith Perry, and other originals [7:38] and the idea of the “skeptical optimist” [12:52]
      • The importance of depth of experience [14:41], what it means to be an expert [19:49], and implications for entrepreneurs [21:12]
      • How originals were influenced in childhood [23:42]
      • Applying this research in organizations [31:34]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and today I’m here with Adam Grant, who is a Wharton professor and a New York Times columnist who covers the topics of work and psychology. And he has a new book out: “Originals: How Non-Conformists Move the World.” And we thought it’d be really interesting to talk to Adam about this, because there’s a lot of overlap between non-conformists and entrepreneurs. Welcome, Adam.

      What is a non-conformist?

      Adam: Thank you.

      Sonal: The first thing I want to start with is how you actually define a non-conformist, because I think there’s a whole spectrum — and I use spectrum in both the psychological sense and just a descriptive sense. How do you know if someone’s a non-conformist, and not, like, a good rebel versus a bad rebel — like, someone who’s actually detrimental to society?

      Adam: When I think about non-conformist, I’m thinking about people who don’t just reject the status quo for the sake of being different, or for disagreeing, but actually care about making things better for other people. So, I think about, you know, non-conformity as being — you can think about, you know, creative rebels who say, “Look, you know, there’s a standard way of doing things that isn’t right. And I think I can improve it.” Or, you can think about being a moral rebel and say, “You know, there’s a rule, a law, a policy, that doesn’t make sense, and it’s hurting a particular group. And I want to try to do something about that.”

      Sonal: So the constructive form of non-conformism, essentially.

      Adam: Exactly.

      Sonal: One funny anecdote that you mentioned, which made me laugh out loud, is that a person’s choice of browser indicates where they fit. So, if you’re like a Chrome user, for example, or a Firefox Mozilla user, you’re more likely to be along those lines than not — it feels kind of obvious in hindsight, but it’s a really funny thing to come across. Like, how did you sort of come up with that?

      Adam: Well, I wish I could take credit for it. I was sitting at a conference one day and an economist, Michael Housman, presented this study showing that he could predict your job performance and how long you stay in your job just by knowing what browser you use. And a lot of people don’t like the results of this study, obviously. And if you don’t know what browser you use, you should check Ask Jeeves right away. Basically, what he found was, Chrome and Firefox users were on average getting, in call center jobs, to customer satisfaction rates in 90 days that took Internet Explorer and Safari users 120 days to reach. And the Chrome and Firefox users also stuck around 15% longer in their jobs. So, the first instinct for me was, this has got to be a technical advantage, right? The people who are more computer savvy are the ones who are using Chrome and Firefox.

      Sonal: That’s actually what I would think too.

      Adam: But Mike ran the data, and there was no difference in typing speed or computer knowledge between the different browser groups. And what I eventually realized was, it’s about how you get the browser. Because if you’re an Internet Explorer or Safari user, those came pre-installed with your computer. They’re the default, and you just accepted the status quo that’s handed to you. Whereas, if you wanted Chrome or Firefox, you have to take a tiny bit of initiative and upgrade, or figure out, you know, maybe there’s something different out there. Let me try it out. And that turns out to be a signal of being the kind of person who doesn’t just conform and accept the defaults that are given to you.

      Sonal: What are some of the other counterintuitive characteristics? I think, by the way, procrastination was super interesting to me.

      Adam: I think we’ve all procrastinated at some point in our lives. And I will say though, I’m pretty much the opposite of a procrastinator. There’s a term for me — I’m a precrastinator. So I’m one of those people who, when I have a presentation to give in six months, I will wake up tomorrow morning feeling this tremendous sense of urgency to get it done now, so that I don’t wait until the last minute and it’s not hanging over my head. And as I studied originals, I found that many of them resisted that temptation and use procrastination productively. 

      So, look at Leonardo da Vinci, for example, who spent roughly 15 years trying to finish up “The Last Supper,” and kept putting it off and working on these little optics experiments. And he felt like he was spinning his wheels, and wrote in this notebook over and over again, “Tell me if anything ever was done.” But ultimately, those diversions led him to make these discoveries in how to display light that dramatically improved his painting and made him the Renaissance man. And there’s a lot of research coming out suggesting that when we procrastinate, we give ourselves more time to incubate ideas. We do more divergent thinking. We’re less likely to be stuck in linear structured patterns of thought. And that can be useful if you want to come up with new ideas.

      Sonal: I think I would probably nuancify the description of procrastination then, a little bit, because I think you either do it or you don’t. And I think what you’re actually saying is actually that there’s something more in between. Because when I think of how I procrastinate as well, you know, it’s not like you’re not working on it. You’re working on the back of your head, or you’re putting on the back burner, or you’re exploring ideas that are related to X topic. And then, when they suddenly kind of come together, you’re like, “Okay. Now is the time to actually put it together on paper, and this is the right time to put it out.” There’s another form of not procrastinating, where you just stick your head in the sand, and you just completely avoid it. Like, the kind I remember doing, like, in high school — you don’t even show up. I feel like there’s different flavors to those kinds of procrastination in what you’re describing.

      Adam: Yeah. I think productive procrastination is intentionally delaying the start or finish of a task to make sure you have all the creative ideas that you might develop at your disposal. And that’s very different from just not engaging with the task at all, right? So I had a Ph.D student, Jihae Shin, who studied this, and she found that there’s a curvilinear relationship between how often you procrastinate and how creative your supervisors rate you in multiple organizations. So, if you put things off until the very last minute, you’re screwed, and you just have to rush forward with the easiest ideas. But there’s a sweet spot, where you put things off a little bit, you’re delaying, but you are kind of doing some unconscious thoughts, some incubating. And that allows you, then, to come forward with more interesting ideas and more unusual possibilities, because the first ideas that you generate are usually the most conventional and obvious.

      Sonal: Right, exactly.

      Adam: And if you just marched forward with those, then you’re limiting your field of vision. But it is key that you are, sort of, processing the task. So one of Jihae’s experiments randomly assigned people to procrastinate before developing business plan ideas by playing “Minesweeper.” And they were, after doing that, 28% more creative than people who jumped right into the task. “Minesweeper’s” awesome, but it’s not the reason to become more creative. The effect only held if they were told about the task before they played “Minesweeper.” So that, you know, while they’re working in the games, they’re kind of thinking about different business ideas. And that’s where the creativity came in.

      Sonal: So what’s interesting about what you’re describing is a type of behavior. But in the real world, people have deadlines, and constraints that they have to follow, and things that they have to deal with. Like, if you’re in a company, or in school, or, you know, in just paying your bills on time. How does this sort of behavior play out in those scenarios? Like, can you be an original in one aspect of your life and then suddenly be very punctual about paying your bills? The psychological traits we have — it’s not like you get to pick and choose what arenas of your life you get to be a certain way.

      Adam: Yeah. I think it’s hard to be an original without being flexible. In fact, that might be the most central defining characteristic of original people, is that they’re willing to bring different ways of solving problems to different situations. So, of course, there’s some tasks — actually, take any task where creativity is not important. If you’re paying bills, you don’t have to come up with novel solutions. And in fact, if you do, the IRS might come calling. That’s a task where you want to be structured, conscientious, focused, and, you know, sort of punch it in as quickly and efficiently as possible. I think that where originals end up procrastinating is when they know they’re working on a hard problem. One of my favorite examples of this is Aaron Sorkin, the screenwriter who’s known for the “West Wing” and the Steve Jobs movie. And he was interviewed once by Katie Couric, who said, “You know, basically, you drive your staff crazy, because sometimes you’re literally about to shoot a scene, and there’s still no script. Like, how do you put up with this procrastinating?” And Sorkin said, “You call it procrastinating. I call it thinking.”

      Case studies

      Sonal: It’s an interesting way of reframing that. I know you were talking about Aaron Sorkin writing the script for that movie, but I do think it’s interesting to talk about Steve Jobs as an example here because — and I definitely don’t want to be one of those people who elevates to this cult of Steve Jobs all the time. I think we need to be both critical and mindful of what he did and didn’t do. But one thing that struck me when I was reading his biography, the one that Walter Isaacson wrote, was this concept of his reality distortion field. And I think it’s very closely tied — like, you have to have some sort of flexibility of reality, a view of reality, in order to distort it for a better world and be able to envision a better possibility. But then there were times when it became just straight-up delusion. How do people navigate that balance? And what are your views on how this sort of thing played out with an example like Steve Jobs?

      Adam: It’s a really interesting question. I feel like Steve Jobs is a Rorschach test, where you put him out there, and then whatever response you hear is much more revealing of the person answering than it is of Steve Jobs.

      Sonal: Oh, that’s so interesting. That’s actually a really good point.

      Adam: So, what does it mean that that was my reaction? I don’t know.

      Sonal: That’s fair.

      Adam: No. I mean, okay. So I’ll put my biases on the table here. I think that, for me, successful originals are not distorting reality as much as they are choosing when to present different realities. My favorite example of this is Meredith Perry. So Meredith has this amazing startup called uBeam, which is doing wireless power, right?

      Sonal: Full disclosure, we’re actually investors in that.

      Adam: When Meredith came up with the idea for wireless power, she went to some physicists and engineers, and they all told her it was impossible and she was insane. And she was in this, sort of, chicken and egg, catch-22 scenario, where she needed to build a prototype to prove it. But she couldn’t get anyone to work for her, because they all told her she was out of her mind. And at some point, she realized that instead of going to engineers and saying, “You know, I’m trying to build wireless power. Can you create this kind of transducer for me that I think will help me convert vibrations in the air into energy?” and having them say, “No, that can’t be done,” she started hiding her purpose and telling the engineers she was trying to recruit about the means that she wanted but not the ends. So, instead of saying, “You know, I’m trying to build wireless power, you know, I needed a transducer that will help me convert vibrations in the air into energy,” she just said, “Do you think you could build me a transducer with these properties?” And all of a sudden, instead of, “Hell no,” the answer was, “Yeah, I could probably figure out a way to make that work.”

      And I think this is such a good example of timing which realities to present, as opposed to distorting them. It’s not like she’s lying to them. She’s not saying, you know, “I’m trying to build a transducer in order to tie my shoes faster.” She’s just choosing to reveal this information after she has more of the technology available, and people will be much more likely to believe her. And I think this is, to me, a fascinating strategy for originals, because we’re always told, especially if you watch Simon Sinek’s TED Talk, start with why.

      Sonal: Oh, I love that TED Talk.

      Adam: I do, too. And I think the point is largely right, that you have to — in order to motivate people to come on board with most ideas and visions, you have to explain your purpose. But if you have a really original idea, that’s terrible advice. Because your “why” sounds insane to other people. And so, if you’re Meredith, or if you’re Steve Jobs, for that matter, sometimes the “how” is much more believable than the “why.” And I think that that’s a skill that we could all work on, right, knowing when to say, “This is my ultimate goal,” and when to say, “You know, I’m kind of working toward this mid-level objective. Do you think you could help me with that?”

      Sonal: The most fascinating aspect of the anecdote you just shared, to me, is that she essentially had a very original idea, and had to take a very non-original approach. She had to use people’s non-originality in order to get them to deliver what she needed incrementally to get her to the next goal. So it’s almost like, you have this interesting interplay in an organization between the people who have these characteristics and people who don’t, and then how you, sort of, interact with each other and how you switch contacts based on that.

      Adam: Bingo. And we see this with lots of great entrepreneurs. This is exactly what Elon Musk did with SpaceX. He didn’t recruit his team by saying, “Let’s go to Mars.” He said, “Let’s see if we can get a rocket into orbit, and then back.” And once they saw that was possible, it’s a little bit more acceptable to start talking about whether we can colonize another planet.

      Sonal: But don’t you want people who believe in your vision? Because when you say, like, Elon Musk recruited those — some of those folks, I mean, okay, clearly, he needed people who aren’t just so pie in the sky that they actually need to build what he’s envisioning. But at the same time, I feel like one of the defining characteristics of startups, and sometimes for really good leaders of startups, is this collective of people who believe in a similar vision. And not to sound, like, cult-like or like it’s a mission, but more like — it’s a way to really align people around like you’re doing something. Like, I know I would not want to work for someplace where I don’t believe in the product, for example.

      Adam: Of course. I think, though, that the people that I would want to hire, and that I think Elon wants to hire are skeptical optimists.

      Sonal: Let’s break that down a little bit.

      Adam: Yes. So the optimism part is, you believe that the future can be better than the present and the past. And when you consider possibilities, you’re willing to have hope and see upsides. The skepticism is saying, “I’m not going to be convinced by every Pollyanna idea that somebody throws at me. I want to see the hard evidence, you know. Show me that this is doable.” And so, you know, I think actually recruiting people who think that they could do, for a fraction of NASA’s budget, something that NASA has never been able to do — those are people who are willing to believe in a vision. That buys you the optimism. But I don’t necessarily want to recruit people who believe on day one that Mars is a realistic destination in the next decade or two.

      Sonal: I love that you said that phrase, of skeptical optimism. Chris Anderson, the former editor in chief of WIRED, used to always kind of describe our mission when we were at WIRED as being informed optimists. Like, the bar for a story getting through was, like — it’s not only optimistic about the future of technology, but that there’s a level that it’s informed. It’s not just conspiracy theory. I love that. And I have to say, by the way, though, Adam — I think that you’re not the only one doing this, but a lot of people use Pollyanna as this way of saying Pollyanna-ish ideas. Like these, sort of, fluffy, idealistic, feel-good, rose-colored glasses, view of the world. And in reality, Pollyanna is actually a story about overcoming hardship. The reality of the movie is — and I hope I’m not giving spoilers, because that movie is like 50 years old, but — is that she has a very optimistic view of the world, but then she gets paralyzed. And she has to then overcome her own fear of her own limitations in order to hopefully overcome that disease.

      Adam: The movie is very much about the importance of a certain kind of optimism for overcoming tragedy. I think that, obviously, the modern use of the term has evolved to, sort of, focus on “your glasses are so rose-colored that you might be a little bit unrealistic” or “too easily duped — like, you’re gullible.”

      The meaning of “expert”

      Sonal: And that’s totally fair. I’m just, like, putting out my own personal agenda to defend the movie. Let’s talk a little bit more about some of the more counterintuitive characteristics. What are some of the surprising things that define originals, based on your research in this book?

      Adam: I think one of the things that really caught me off guard is that they tend to have less expertise than a lot of their peers.

      Sonal: Ooh, interesting.

      Adam: Yeah. So there’s this curvilinear relationship between expertise and originality, where when you’re trying to come up with new ideas, you obviously need to know a field or a domain well enough to have an understanding of what’s possible and what’s been done in the past. Right? So, Einstein couldn’t have come up with his theory of relativity without knowing something about physics beforehand and studying Newton. But it’s not a coincidence that he was relatively new to the field, because the longer you learn a particular domain of knowledge and the more you internalize it, the easier it is to become entrenched, where you basically take for granted assumptions that need to get questioned.

      And what you see with a lot of successful originals is they have this great combination of breadth and depth. Where, yes, they know the domain reasonably well, but they’ve also immersed themselves in ideas outside that domain to make sure that they’re seeing things from a fresh perspective. One of the ways you see this is — actually, if you look at Nobel Prize-winning scientists. One of the things that differentiates them from their peers is they’re much more likely to have artistic hobbies. So, on average, Nobel Prize winners are twice as likely to play a musical instrument. They’re seven times as likely to paint and do other kinds of art. They’re 12 times as likely to write creative fiction, poetry — and they are 22 times as likely to act, dance, or perform magic.

      Sonal: Oh, my God, that’s actually really funny.

      Adam: Yeah, as a former magician, I love that stat. But…

      Sonal: I think Aaron Levie, the CEO of Box, is actually a magician too.

      Adam: That is right. And I think that, you know, obviously, this is not all causal. The same curiosity that draws people to be creative also tends to pique their interests in these kinds of artistic hobbies. But sometimes, engagement with these hobbies helps with the discovery of original ideas. Einstein said that the theory of relativity was a musical thought, and that it came to him because of all the time he spent playing the violin. And Galileo — one of his greatest discoveries was being able to spot mountains on the moon for the first time. What’s remarkable about Galileo is, he was looking through his telescope at an image that other astronomers had seen, but he was the only one who recognized that the shading he was observing was mountains. And the reason for that was, he had specialized in a drawing technique that used a very similar kind of shading. And so he knew that that was how you represent a mountain. And in that case, if he had not been an artist, he never would have made that discovery.

      Sonal: So, that’s a case where the art actually influenced how they viewed certain things. So, I mean, you’re describing two things. One is this, sort of, co-occurrence of this creativity in a field or experience in a field. But you’re also describing something where you’re talking about the exact right amount of experience and expertise, and you’d mentioned it as being curvilinear. And that’s the second time you’ve mentioned the curvilinear as an example. So, it sounds like there’s always, like, a sweet spot — where there’s not too much, not too little, but there’s just this one — just right amount. How do you know what that sweet spot is? Like, where do you sort of fall off into one side of the curve or not?

      Adam: I think that part is much more art than science. As much as it pains me as a social scientist to admit it. I wrote a paper about this a couple years ago with Barry Schwartz, where we argued that everything in life is an inverted U. And that, you know, if you take any strength, or virtue, or positive experience, you can find too much of a good thing — where, you know, like, okay, if you’re too confident, we get narcissism. If you’re too generous, we get altruistic self-sacrifice. And you can play this out for any trait that you probably see in a positive light. I think the only thing that we really know at this stage about how to find the sweet spot is that good things satiate and bad things escalate. So, the further that you move down the positive end of something, the more likely the costs are to start outweighing the benefits. And I think you can only usually see that by looking at the results. So, you know, in the case of expertise, right, the question is — okay, when you start to generate ideas, are you finding yourself trapped by what you already know in the field? As you’re, you know, evaluating different kinds of ideas, do you consistently gravitate toward what’s already accepted and proven?

      Sonal: It’s really funny that you guys tried overfitting the U-shaped curve to all these different things as well. But I think that there’s a gender or racial background, or other background effect that can play out here differently. I’m thinking of cases where a lot of women, myself included, will sometimes underplay their expertise. Because, you know, I’ve seen a lot of my former male colleagues — like, they would be experts in things that they necessarily weren’t. But they have the confidence to say that they were. And to me, that wasn’t a sign of confidence. I’d actually get really irritated when people said, like, “Be confident and say you’re an expert in that.” And I’d be like, “I’m not going to frickin claim to be an expert on something I’m not. Like, I don’t think that’s confidence. I think that’s just being full of crap.” I think it’s really interesting, because I think some of these things also play out where there’s an interaction effect between people’s background, whether it’s gender, race, or privilege, or other things that have influenced how they grow up. Like, how did you see that play out in thinking about originals?

      Adam: Yeah, it’s interesting even how you set up that comment, right? Because a man would have said, “It’s a fundamental fact,” as opposed to, “Here’s what I think and I’ve kind of noticed…”

      Sonal: Oh, God, you’re right. It’s funny because when I hear myself on the podcast, I’m always like, I really got to take out some of those caveating words I use, like kind of, maybe, what do you think, vocal fry, whatever. All the stuff that I think I tend to do sometimes, and I hear it and I cringe. Other times, I’m like, “Fuck it. It’s who I am, like, take it.” You know what I mean? Like, anyway, you’re totally right though. So…

      Adam: Well, let’s take that a step further, though.

      Sonal: Yeah, let’s talk about it.

      Adam: Do you really want to take that out? I would say maybe not. So, Zak Tormala at Stanford has these studies showing that experts are believed more when they express uncertainty.

      Sonal: I like that, because I actually think that is what a true expert is. It’s hubris to claim to be an expert in something you’re not. At the same time, it will say, coming full circle to your point, I have observed that the people who actually go out and start companies — and this takes us to entrepreneurs — are people who have such belief in an alternative view of the world — even if they’re not experts in X, Y, or Z, that they’re the ones who go out and do it. And I admire that. I do think it takes a certain amount of knowing that you can do that. So, like, what’s the difference? Is that a confidence thing? Is that an experience thing? Is that an original’s mindset? I mean, where do we figure out, like, what’s making an entrepreneur tick there?

      Adam: There’s a huge debate about, you know, how much does confidence really drive success. And, like everything else, is curvilinear. Right? So, if you have too little confidence, you never act. And if you have too much confidence, then you end up getting complacent and missing out on threats and opportunities that you underestimate. I think that where I would come down on this is — Susan Cain is fond of saying that there’s zero correlation between who’s the best talker and who has the best ideas. And that’s true empirically. The sad thing is though, a lot of us take confidence as a signal of competence. And I think we need to stop doing that. I think if we stop doing that, we’ll see many more women and minorities rise into leadership positions, because we’ll see that oftentimes they are better prepared, but communicating in such a way that didn’t always signal, you know, the confidence behind the idea.

      I would also say that the self-esteem movement has been disastrous for entrepreneurs, in the sense that, like, becoming a successful entrepreneur is not about thinking that you’re special. It’s about believing that, you know, somebody else could do this. Maybe I could too. And I think that confidence should come as a consequence of competence. Right? So instead of saying, “Well, I need to build my confidence, and then I’ll be successful,” no, let’s develop grit. Let’s have a growth mindset. Let’s work as hard as we can to achieve success. And then confidence will be the natural product of that.

      Sonal: When you know something really well, or you feel very passionate about it, it feels true to you. The way I feel and, say, personally about editing — you feel incredibly confident in that, because it is a consequence of competence. You quoted Susan Cain’s work, and she’s the author of “Introverts.” That — there not necessarily being a correlation between how one communicates and that confidence, and the competence. I think it’s interesting, though, because — when it comes to leadership, as you know, and I’ve seen this with entrepreneurs as well — you are motivating people by being able to communicate your vision as well. And it has implications for hiring, for everything else. So, it actually really does matter. I mean, I don’t think we can easily dismiss that out of hand either.

      Adam: Let’s be careful not to overrate confidence. But, yeah, it plays a role in our lives. I think for most people, grounded confidence comes from accomplishments, not the other way around.

      Sonal: Are there any other takeaways? Like, I’m actually curious, not just for entrepreneurship, but like, for education, for raising kids. Like, you had an article in the New York Times this past weekend that talked about, like, the mistakes a lot of parents commonly make — like, you know, over-programming their kids. I mean, what are some of the implications of your research?

      Non-conformists in childhood

      Adan: One of the things that surprised me the most is, when you study originals and look back at their childhood histories, you see that their parents often focused on creating a really strong moral compass. So, there are these brilliant studies of creative architects, where you look at the people who are nominated consistently by the most respected people in their field as truly original. And then you compare them to their peers, who are technically skilled, but haven’t necessarily done anything creative. And in the original studies, there were extensive interviews — not only with the architects but their family members, lots of observations, assessments. One of the things that came out was that the parents of the creative architects tended to focus less on rules and more on values. But that they gave their children a lot of freedom to actually determine their own values. And what happened was, the architects, then, develop their own values, which, you know, were grounded in a moral framework — that when other people said your idea was ridiculous, they were much more comfortable standing their ground and saying, “Well, this is who I am. And I’m going to try it anyway.”

      They also, you know, in addition to just being comfortable with nonconformity, they were much more likely to be concerned about, you know, what is my contribution to the world, right? When I’m constantly asked to think about what are the consequences of my actions for other people, I want to leave the world better than I found it. And I think we could probably all do a better job. I know I can as a parent — you know, really having that conversation about, you know, here’s some broad values that we think matter. How do you want to live by those?

      Sonal: Are those things you think communicated verbally or through modeling? Because one thing that comes to mind when you describe that is, that probably explains a lot of immigrant children’s success in the first, second, and third generations. I know the effect tends to disappear after the third generation in past studies. But I wonder, with immigrants, it’s kind of, like, this epic that you get, because you just watch it. I mean, I wouldn’t speak blandly for every ethnicity out there. But a lot of immigrant groups — you’re not having those conversations with your parents in any kind of articulated way. Like, it’s not a — it’s a very nonverbal type of culture in that way. And so, you can actually learn those things just by watching them work hard and try to contribute something to the world.

      Adam: Yeah. Modeling effects are often stronger than conversation effects. In part because, you know, role models — when you see the behavior, it teaches you how to do it. It tends to raise your expectations of what’s possible. Conversations don’t always have that impact. They also sometimes, you know, create this reverse psychology reaction of, “Don’t tell me what to do, I will now do the opposite.”

      Sonal: Exactly. You also mentioned role models, the existence of. Because one thing that comes to mind as well — and I’m thinking of classic developmental psychology studies of resilience in orphaned children [who] were orphans in previous world wars. And one of the consistent findings that came through over and over again is, no matter what else those children did not have, one of the greatest predictors of resilience was having a person they could look to as a mentor or as a role model. How does that play out with — taking it a step further — like, beyond survival, to becoming a productive non-conformist?

      Adam: I think that parents don’t necessarily have to be role models. I think that everyone needs a role model in order to have some kind of vision for what it looks like to make a mark. But we can find role models in some pretty unexpected places. There’s some classic research looking at patent rates and innovation trajectories of entire economies. And my favorite finding out of this body of work is that you can predict the spikes and falls in U.S. patent rates by coding themes of original accomplishment in children’s books.

      Sonal: Really?

      Adam: Yeah. So if you look at the children’s literature of a particular era, when that literature starts to include examples of people accomplishing things and succeeding in ways that are new and innovative, patent rates actually spiked 20 to 40 years later.

      Sonal: Was the era where Dr. Seuss, like, published “Cat in the Hat,” like, super high on patents? Like, I mean, those kids were fully defying their parents with, like, the cat mouse.

      Adam: Yeah, I think there’s a case to be made there, and more recently “Oh, The Places You’ll Go,” I think was the most popular children’s book in the ’90s, which was all about choosing your own path. And what’s fascinating about this is, you know, in part is just a reflection of the culture. Right? So, you know, when innovation becomes more important, we tend to write, and buy, and read children’s books that are innovative. But there’s, I think, a story to be told about how these books actually shape originals. What you will find is that if you talk to some of the great originals of our time, they are constantly saying their favorite books as kids were stories of, actually, other kids who were, you know, inventing things, or accomplishing things that were impossible previously. If you ask Peter Thiel and Elon Musk to name their favorite childhood books, they both pick “Lord of the Rings.” Jeff Bezos and Sheryl Sandberg both said “A Wrinkle in Time.”

      Sonal: Yeah. I used to love Madeleine L’​Engle.

      Adam: Yeah, we all did. Right? And what is that story about? It’s about a young girl bending the laws of physics and traveling through time.

      Sonal: Yes.

      Adam: Like, if that doesn’t get you thinking about making an original contribution to the world, I don’t know what does.

      Sonal: Well, they’re both sci-fi books. They all fall in the sci-fi genre, which is interesting in and of itself.

      Adam: They do. And it’s not a coincidence, by the way, that you can trace a lot of modern inventions to, you know, the writings of Jules Verne and the technology that we watched on “Star Trek.” And, you know, I think in a very real way, these fictional role models give children the freedom to define their own niches, and imagine doing things that don’t exist or aren’t currently considered possible. 

      I really think “Harry Potter” is going to have this impact. I would probably put my money on J.K. Rowling as the most influential original alive, because “Harry Potter” sold more books than any other series except maybe the Bible. So it’s reached a lot of people. You have kids saving the world and inventing spells in ways that spark lots of creative thinking. And there’s also academic research now showing that, after kids read “Harry Potter,” they become less prejudiced. So, they learn not to stereotype people in the way that wizards look down on muggles. And so, you know, that’s a pretty good trifecta, right, reaching hundreds of millions of people, getting them to think in original ways, and making sure that they don’t have these strong in-group, out-group boundaries.

      Sonal: I’m fascinated by that. I mean, it’s one of my all-time favorite series. So, I’m personally incredibly motivated by “Harry Potter.” But to hear that it can have that effect on people is incredible. And to your point, it does indicate how culture does shape the sort of thinking that comes out in each generation. What’s more fascinating to me is the recent uptick in the last 5 to 10 years of young adult literature that is — really strong female characters. You know, like, obvious examples include “The Hunger Games” with Katniss Everdeen, the “Divergent” series by Veronica Roth. And there’s, like, countless others. I mean, I read, like, one a month. They’re just amazing.

      Adam: I’m glad I’m not alone.

      Sonal: No, and you’re not alone. And what’s actually refreshing is when we were growing up, do you remember even having that many female strong characters out there? Because I don’t remember that. I used to read stuff like — I mean, I remember reading like David Eddings, like, “The Belgariad,” or, like, other things that — they were male characters. There weren’t, like, strong female characters, or if there were, they were, like, adjuncts to the male character instead of the main character.

      Adam: Yeah. As I think about it, like, the female protagonists or heroes, like, in my childhood were Nancy Drew, Wonder Woman, and maybe Penny from “Inspector Gadget.”

      Sonal: Exactly. And they’re great strong women. But it’s very different than today, where you have Katniss Everdeen, like, murdering people for survival in “The Hunger Games” arena. I mean, they’re strong, hard characters. And I think that’s incredible. One other interesting thing, though, is I think we’re only talking about books as an influence on literature. I think it’s important to mention other forms of narrative, like TV shows. Things like “Game of Thrones” where there’s really no narrative arc. It’s this endless story that keeps developing. Or even, like, video games. Not all video games have a fixed content arc if they’re not a content-based video game. So, I think that’ll be interesting to see how that plays out in what you’re describing here, because there might not be as many examples in the future of that sort of thing.

      Adam: I think so, too. I’d love to see the evidence.

      Implications for organizations

      Sonal: So far we’ve been talking about a lot of interesting themes, and research, and anecdotes that are really centered on outliers as cases, or as individuals. One thing I’m interested in, particularly because you’re a professor at Wharton, and you studied management science and things connected to this. How does this all play out systematically, like in the organization, for example? Because we all live — like, we probably spend more of our day inside a firm than we do in our own families. And so, I’m really interested in hearing how these dynamics play out in groups, and culturally across organizations as, sort of, containers have that sort of culture.

      Adam: Part of “Originals” is about how individuals can champion new ideas, and then how parents can try to nurture kids to think differently. But I think it’s just as important, if not more so, for us to understand how leaders build cultures that welcome original thoughts and that fight groupthink. And there are a couple of things that most leaders do wrong, if you look at the data. First thing is, hiring on cultural fit. So, one of my favorite studies looks at over 200 Silicon Valley startups and tracks them before and after the dot-com bust.

      Sonal: I like where this is going. This is going to be interesting.

      Adam: Yeah, this is fun. So, you see that there are three prototypical ways that founders hire when they’re looking at talent. Some hire on skills, so they’re looking for people who have a certain set of competencies now. Some hire on potential. So, it doesn’t matter what you know today and — how much do we think you can learn? And then a third group hire on cultural fit. Do you share our values around here? And when you track the founders’ firms, what you see is that the founders who hire on cultural fit are less likely to see their startups fail, and they’re more likely to make it to IPO. And then after IPO, their firms grow at a much slower rate than the ones that hire on skill or potential. So, cultural fit helps you grow and take off, and then it causes you to stagnate, and maybe increases your risk of failure. Why is that?

      The basic explanation is that cultural fit is a great way to get groupthink. You have these founders who are incredibly original at the outset, and then they hire a bunch of people who see the world exactly the same way they do, and they end up cloning themselves, and getting homogeneity of thought instead of diversity. So, I think the solution to this is, at some point, as your organization grows and you need to start questioning the very values that made you successful in the first place, you want to stop hiring on cultural fit and start hiring on cultural contribution.

      Sonal: Ah, so what’s the difference there? Because I mean, I think people would — I would conflate that.

      Adam: Yeah, so cultural fit is, basically, “Who are we, and how do we bring in people who are just like that?” Cultural contribution is asking, “What’s missing from the culture, and how can we bring that to the table?” So, trying to figure out who’s going to enrich the culture and add diversity of thought to it, as opposed to just replicating it. Of course, you can also overcome some of these problems if diversity is one of your core values.

      Sonal: Right. I agree. I don’t think you can just tack it on, you know, as, like, a sidebar silo thing, it has to start from the leadership. It has to start from the top, like, you have to believe in something. You have to believe — whatever values leaders believe in is essentially what the company is going to believe in.

      Adam: Yeah. And I think, you know, you can screen on this in really interesting ways. So, one of my favorite interview questions is to ask people, “Tell me what’s wrong with our interview process and how you would improve it.” Or, you know, more broadly, “Based on what you know about the culture so far, if you were in charge here, what are the three biggest changes that you would make?” People aren’t willing to give that kind of constructive criticism or bring in dissenting opinions in the hiring process — I’m a little worried about whether they’re going to do that moving forward.

      Sonal: One thing I’m really fascinated in — because you brought this up a number of times in the book — is power differentials. And that’s power differentials between people, like, speaking truth to power, people have less powers, people who aren’t in management but who are contributing original thinking — even power differentials with people who are not represented, like, whether you’re underrepresented in the organization. How do those play out in this scenario?

      Adam: Unfortunately, the evidence suggests that lots of people who come from non-dominant groups are less likely to get heard when they speak up. So, Sheryl Sandberg and I wrote an op-ed last year called “Speaking While Female,” where we covered a lot of evidence that when a woman and a man make the same point, the man gets a big pat on the back and people start to rally around him. And the woman is either not heard, or punished for being too aggressive. And you see the same effect with different kinds of minority groups, because this is not fundamentally a gender effect. It’s a power effect. Right? So the groups that are perceived as, you know, as not occupying high-status positions in society get stereotyped as, you know, sort of needing to find their place. And, you know, they’re often perceived as stepping out of bounds when they’re just trying to make suggestions or get their opinions heard.

      You know, I think, from an organizational standpoint, we need to be especially careful to welcome dissenting voices when they come from people who don’t look like everyone else, and who don’t come from the most common backgrounds in the organization. But, I think, from an individual perspective, one of the opportunities to overcome these biases is to make sure that you earn status before you exercise power.

      Sonal: That’s a good way of putting it.

      Adam: Yeah. Once you’re recognized as an expert, an authority, as, you know, having made valuable contributions to the organization, it’s much easier for people to look at your suggestion and say, “Yeah, you know, like, you’ve given a lot here, so you have license to deviate from the majority’s preferences.” Or, you know, “You’ve shown that you really care about the group, and you’re committed to making the team successful. And now we’re going to interpret your idea as, you know, an effort to help us get better, as opposed to a threat or a challenge.”

      Sonal: Of course, that does, by the way, assume a very meritocratic organization, because there are plenty of cases where there’s a very nepotistic flavor of earning status. Like, you happen to work with someone before and you believe in them, or you’re just friends and buddies, and you guys party together, or — you know what I mean, like, not everything necessarily plays out in a meritocratic way. There is that effect as well, I think.

      Adam: So true and so sad.

      Sonal: Yeah, it is.

      Adam: You know, I think one of the other opportunities for leaders on this is, when making decisions, almost every leader I’ve ever worked with has made a point of assigning a devil’s advocate, and said, “You know, look, we need to make sure we have, you know, divergent thinking in the room. We want to hear all the dissenting opinions. So, you know, I’m going to ask a few people to represent the opposite.” The sad thing is, if you look at 40 years of research by Charlan Nemeth at Berkeley, she shows that devil’s advocates rarely work. You know, when you’re given the devil’s advocate role, you don’t argue as forcefully as you should. You’re like, “All right. So I’m going to take the opposite perspective. Okay. Now, let me go right back to what I really…”

      Sonal: Or you can have something like the New York Times, where you have like devil’s advocate. I mean, I don’t — I know you work there, so you may or may not be able to comment on this — but where you have, like, someone like Margaret Sullivan, who plays the role of public editor, and I love reading her stuff. It’s fascinating. But even though it’s public, and it’s a criticism of the New York Times, it’s, like, a siloed thing. Like, do people actually then do anything with those takeaways?

      Adam: I have no idea. But, you know, the evidence would suggest that a lot of people don’t, because the other side of this is — just as you don’t take the role seriously enough, your audience doesn’t either. They’re like, “All right, so we know you’re just playing the role. So, we’re just going to let you give your lip service to it and stick with what we already believed.” So what do you do? What you do is, instead of assigning a devil’s advocate, you unearth a devil’s advocate. It’s only authentic dissent that has the best chance of working. That means you need to find people who genuinely disagree and invite them into the conversation. And guess what? That’s more likely to be minority group members, right, who come from a different perspective and a different background. And, you know, this is one of the things that’s easy to talk about and hard to do, which is — you actually have to know what people think. Right? You have to go out to meet the silent minority and say, “Look, you know, we really value your input. Let’s find out what your reaction is to this idea.”

      Sonal: That’s actually a good way of putting it. There’s so much more we could talk about, but I think people should just go ahead and read your book, “Originals: How Non-Conformists Move the World,” which is out now. And thank you for joining the “a16z Podcast.”

      Adam: Thank you for having me.

      • Adam Grant

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Things Come Together — Truths about Tech in Africa

      Nkiru Balonwu, Alan Knott-Craig, Nanjira Sambuli, and Sonal Chokshi

      We often hear stats like “more people have mobile phones than toilets” about places like Africa, but what does that actually mean for people? “It is b.s.,” (no pun intended!), argues one of the guests on this episode.

      Then there are statistical predictions about mobile penetration and usage — for example, that there will be one mobile phone per African within just three years. But how do we make sense of such stats, in context? It may make more sense to measure household per device(s) not just per person … and whether women, too, truly have such access given power structures. Finally, since access is mainly about affordability, what good is a smartphone with an internet connection if data plans are prohibitively expensive? Watching just one 7-second YouTube video could cost a low-income African family an entire month of groceries. Yet mobile and wi-fi may be collapsing and renegotiating Maslow’s famous hierarchy of needs. Perhaps the truth is at the heart of these contradictions…

      With three experts from various backgrounds and regions of Africa — Nkiru Balonwu of international music media company Spinlet; Alan Knott-Craig of free wi-fi non-profit Project Isiszwe; and Nanjira Sambuli of Kenyan startup incubator iHub — we explore the nuances of what connectivity in Africa really means. How does this change app design? What does it mean for doing business with Africa? What does it mean for businesses in Africa trying to compete with Silicon Valley (is there really local advantage)? All this and more in this episode of the a16z Podcast.

      photo: David Mutua

      Show Notes

      • Discussion of Africa as a collection of countries and regions, not as a monolith [1:40]
      • The importance of mobile phones and issues with internet access [5:42], including shared devices and types of content that are popular [14:09]
      • Africa’s growing middle class [20:06], issues with apps and data usage [25:21], and the rise of WhatsApp [29:30]
      • Discussion of local advantage when doing business in Africa [34:58] and women in technology [40:46]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast,” and today this podcast is all about technology and Africa. Now, that’s a really huge topic, so we have experts from various backgrounds and regions of Africa to help us cover a lot of interesting nuances behind the stats that we typically hear, as well as a lot of the buzzwords that we commonly hear.

      Our three guests to help us do this today are Alan Knott-Craig who runs Project Isizwe, an NGO based in South Africa that helps African governments get free Wi-Fi in poor communities. Their goal is to create internet access as a utility. Prior to founding Project Isizwe, Alan was an entrepreneur who ran Mxit, which is one of the largest mobile social networks created in Africa. Nkiru is our next guest, and she is the CEO of Spinlet, a digital media company that focuses on African-centric music and has music available on iOS, Android, and also via web browser, and we’ll talk more about why — about that later. Spinlet is headquartered in Nigeria, but they’re also elsewhere in Africa and also in Europe, and in the United States. Fun fact, Nkiru was actually an IPO lawyer who joined the company as general counsel before becoming CEO, and she actually didn’t think she was ready to be CEO until she read Sheryl Sandberg’s book, “Lean In,” which actually inspired her to go ahead and take the CEO job, and now she’s been in it for the past year and a half. 

      And, finally, we have Nanjira, the research lead at iHub. iHub is an incubator, not an accelerator, although it does lead to acceleration of startups. It’s based in Kenya and provides a physical space for connecting entrepreneurs, a way to test their ideas, provide info to them, and much more. And they do a lot of interesting research as well, and some of that will come up in this podcast. 

      Understanding the diversity of Africa

      Okay, so that’s our guest on today’s pod, and we’re honored to have you all join all the way from various parts of Africa. I think the first thing I wanna start with is, sort of, just this notion of — when we talk about people talk about technology in Africa, the first notion I wanna just jump right into is actually de-homogenizing what people mean when they say Africa, because Africa is clearly a huge continent, but we have a tendency to refer to Africa as, like, one big place. And, you know, I don’t have any personal experience in Africa. My mom was born and raised in Uganda, and I studied African literature in — particularly Nigerian authors, like, Flora Nwapa.

      Nkiru: Yeah.

      Sonal: Yeah, Chinua Achebe and others, Wole Soyinka. But, anyway, what I’d love to hear from you guys first is how would you, sort of, define, sort of, what you consider some of the universals when people refer to Africa, and then what are some of the more, you know, things that people need to pay attention not to clump in together when they’re referring to Africa the continent versus, like, different countries.

      Nanjira: The notion seems to be, especially in business, that, you know, buy one get 53 free. So, you know, if you get your business working in South Africa, the model will work in Kenya and Nigeria. And so a lot of business mistakes are being made from that notion, and it’s that sort of actually homogenizing of it. But, yes, there are notions where — I mean, the little cultural nuances that make it seem like a country, but they’re very contextual and very — they’re still emerging. We’re all still learning who we are as a collective. And so, yeah, I hope maybe by 2017, we still won’t have to say Africa — you know, to remind people Africa is not a country.

      Nkiru: I definitely agree with Nanjira and her experience. It’s sort of very similar to mine when people say Africa, I sort of, like, “Oh, there’s 53 countries, and very different experiences.” And particularly when people say, “I’m an expert in Africa.” Or “in Africa.” And I’m thinking, “What does that mean?” I live in Lagos, and I’m not even an expert in Lagos. On Lagos, anything that has to do with Lagos, there’s different people, different kinds of people living here. We’re really, really extremely different. Of course, there’s similarities, like everybody else, but the differences are really, really — they’re quite harsh, the differences. And so when you say you backpacked across the continent for three weeks, and then you then say you’re an Africa expert, it’s a bit irritating from my perspective. But just generally speaking, one of the more — maybe interesting things is when, you know, Americans are doing African voices in film, it’s always a South African accent trying to be Nigerian, or a South African trying to be, you know, a Kenyan.

      Sonal: Right. Well, Alan, what’s your take on this? You were born and raised in Africa, and in South Africa specifically and, you know, interestingly, given that your company focuses on providing free Wi-Fi and working with different governments and communities, you must actually see more of the commonalities across the regions.

      Alan: The way I look at it is, you’ve got, kind of, Arabic Africa, which is very much the Northern part from Sahara up. And then you’ve got sub-Saharan, which is a little bit more homogeneous, and then sub-Saharan you kind of divide it between the West African trading bloc and the East African trading bloc and the Southern African trading bloc, which is — you know, Southern Africa is about, you know, SADC, really — South African Development Community. South Africa, Mozambique, Zimbabwe, Botswana, Namibia. And then you’ve got English-speaking and French-speaking Africa, which are vastly different, kind of, communities. So, it’s impossible to, kind of, lump it all together in one big <inaudible>. And, you know, from our perspective, we kind of — very much, you know, at least from an English-speaking perspective — sub-Saharan Africa is as close to a homogeneous market as you can get.

      Sonal: Oh, okay, that’s actually interesting.

      Alan: But I just wanna say also, you know, there are some stereotypes that the people should know. So, for instance, South Africa is a bit like Germany in Europe, you know. No one in Europe likes Germany. Everyone drives a German car. So, you know, South African product is respected, but South Africans aren’t always the most welcome. You know, Kenyans are well known for talking a hell of a good game, and the guys from Nigeria, they know how to make money.

      Nkiru: Oh, no.

      Mobile phones and internet access

      Sonal: We have stereotypes like that in regions of India as well. Okay, great. So, you guys — so then, what is the major commonality then, because, you know, based on what a lot of the reports that I read about technology in Africa, a big focus is talking about mobile as a sort of this great — I don’t wanna say the word equalizer, but at the same time, like, it does have that sort of a power.

      Nanjira: It’s true. Mobile has really been that technology that has been perhaps most disruptive, and in terms of one of the, sort of, development speak terms being leapfrogging. It’s helped them leapfrogging so many things — communication, access to finances. And this is where we go — again, cue M-Pesa.

      Sonal: Right. M-Pesa as in, the company that came out of — the payment system that came out of Kenya, and it’s basically mobile payments.

      Nanjira: Essentially. Now, the thing is — I think I’d love us to not restrict ourselves to thinking that mobile is the only, sort of, tool through which advancements and developments can happen as far as ICTs go. There’s so many other things that come with the idea of a mobile phone. So, yes, it’s been that technology that has advanced so many things, but now we’re talking about the infrastructure that needs to be underlying access to [the] internet, or access to certain services that are offered via the internet. And so, that also goes beyond just providing through the narrow spectrum of mobile, though that has obviously been the, you know, the front leader, the trailblazer, so to speak.

      Alan: And in terms of commonality, it seems to me that most countries in Africa just are not connected. So, that’s whether it’s roads, or trade, or financial services, or actual, you know, bandwidth. You know, having, personally, you know, tried and failed a number of times. I’ve done okay here and there, but for the most part, any internet startup in Africa is really struggling to, kind of, get any traction on its local market, because there’s just not enough people on the internet. And whilst 3G is pretty ubiquitous, and LTE is becoming even more ubiquitous, the question is not so much about accessibility. It’s a question of affordability. And the vast majority of people in Africa can’t really afford the kind of going rates for 3G. Just to put things in perspective, if you spend — if you watch a 7-second YouTube video on 3G at South African data rates, it’ll cost you about $20.

      Sonal: Oh, my God. That’s a lot.

      Alan: Yeah, that’s a lot of money. It’s, you know, a monthly grocery bill for your average — your low-end household. So, where we’re coming from, you know, after trial and fail here and there, around trying to get more people on to apps and internet applications, decided to go back to square one, and kind of get people on the internet before you try — start selling them things on the internet. And, you know, from my perspective, it just kind of dawned on me that the only way it’s really gonna happen in our environment is if the government starts getting very much involved in infrastructure, subsidizing it, and making it free. And in a country like South Africa, water and electricity is constitutionally a right for everybody. So everyone in the country gets a basic quota of water and electricity, and the government pays for it from tax money. And we’re trying to, kind of, push a model, whereby public Wi-Fi is provided by the government, as well as subsidized by taxpayers, and everybody is entitled to a daily quota.

      Nkiru: The commonalities are that it’s a difficult terrain to navigate, but I think also that’s one of the strengths that we have — is that there’s, like, immense opportunities here in terms of finding, you know, you can sort of create magic out of nothing here, because there’s just not a lot to work with. And so, that’s what I find very interesting, you know, here. And I do agree to — I totally am, sort of, a believer in the internet being — I mean, I sort of — I’m not saying it should be a human right. I’m not that, you know, advanced in saying that, but I still think it should be a right — access to it, anyway, because, from my perspective, what the internet affords is access to the world, access to education, you know, the things that maybe government can’t really provide because, you know, government doesn’t have as much money as it should do, and because of mismanagement. But if people had access to the internet, perhaps, I think that it creates, like, a whole opportunity for people to access information that they then use in, you know, daily life, daily business, health, education whatever.

      Nanjira: I mean, or maybe we should just have it as a right. I mean, maybe we should dare to dream, because I think the internet and access to it and affordability, as Alan rightfully pointed out, are the two key things we’re looking at. But also the fact that participating in society and economies going forward is actually going to be facilitated primarily by the internet. And so, we need to maybe dream and have a bit of a more lofty goal, other than just accepting from our government that there isn’t enough money. The issue, as you’ve rightfully pointed out, Nkiru, is actually that, you know, we’re not poor. It’s actually just mismanagement. So, it would be actually nice to probably dream for that and actually push for that.

      Sonal: I think you guys are completely right. It’s worth actually taking a pause for a moment to reflect on those words, because we tend to throw them around very loosely here in the U.S. — like, access, affordability — but you’re actually talking about true access as a right, affordability as one of the mechanisms for making that happen. And then what you’re really saying, Nanjira, is that there’s an element of inclusion as well — that it’s about getting people the ability to be included in this larger, broader, global movement, where they do have access to this knowledge, information, etc., etc. Well, just to put things in perspective — so, I read a statistic that estimated that the number of smartphones in Africa would be about 930 million by 2019, which is just 3 years away. And that’s basically about one mobile phone per African. How do you guys see that on the ground there? And that, by the way, as we’ve just learned, does not equate with actual internet access. So, what’s the, sort of, differential between having a mobile phone and then the penetration of access to bandwidth?

      Nkiru: So, I can generally just talk about smartphones. Oh, actually, smartphones — I think about 100 million on the continent, considering we’re about 1.1, 1.2 billion now. I don’t know. I’m not quite sure, maybe about a billion. The penetration hasn’t been as fast as people have, sort of, forecasted. Well, almost everybody has a mobile phone, but they’re not smart-enabled, so there’s a difference here.

      Sonal: Right. I’m glad you’re reminding us to make that distinction.

      Nkiru: So, smartphones are not penetrating as fast as we, you know, all the predictions have said. I think a lot of businesses have been predicated on the, you know, the different — what do you call it? The different forecasts about how fast smartphones will penetrate the markets, but they haven’t come as quickly, because they’re really expensive. Of course, with the new — with Androids now becoming much cheaper, we’re seeing penetration is becoming higher, but it’s still not catching up as quickly as we’re hoping.

      Sonal: Okay. So, you think one of the reasons things haven’t penetrated as fast as forecasted — and businesses are being built on these assumptions — is because of the fact that the phones are not quite as cheap yet necessarily for everybody.

      Nkiru: Exactly.

      Nanjira: Yeah. And I think it’s, again, context, you know. Again, just back to the point I made earlier, about the ecosystem and looking beyond the mobile phone as the only tool through which, you know, access should be facilitated. I mean, because — there’s also the question we are asking ourselves now is — in studies that we’re conducting with people at, sort of, the base of the pyramid, or those who are about to be first-time internet users, to better understand their needs — and their understanding of this oncoming device or space that is the internet, just to better understand that. There are needs that go beyond the mobile phone. And so there are those that maybe, you know — we have cyber cafes, for instance, that still exist. And so one, you know, cybercafe could serve maybe 100 people in a locality, and maybe they don’t have mobile phones at home, but they’re still getting access, right? Or, other centers like that. So, there’s a need to start taking all statistics beyond the rush to quantify Africa, as I call it, to take statistics in context and bring them all together to a holistic picture. So, maybe not mobile phones. There may be a plateau, obviously, because of the affordability aspect or that last-mile connectivity element, but what about other ways people are accessing the internet?

      Sonal: So, that’s actually really interesting. Do you have some of those statistics? Or, can you give us a little bit more flavor about what that looks like?

      Nanjira: Right. So, what we’ve found, and it’s really mostly studies that we do, user experience studies — just better understanding the target user for any mobile phone application or service is that, you know, these are very — and especially in a country like Kenya — and I’ll use Kenya as an example here — is much famed for elements like M-Pesa, but there are other cultural factors that also have been hindrances to start that last-mile connectivity. So, you’ll find, for instance, it would be traditionally that women may not have access to a mobile phone, because of how structures of property ownership exist beyond, you know, or predating technology. And so, how to overcome those? And if that lady cannot own a phone at home, maybe she can access the same services she needs on the internet via a center she can go to during the day, when nobody’s bothering her. We don’t know who’s holding what to your head when you’re accessing a mobile phone.

      Sonal: That’s a really good point.

      Nanjira: So it’s — this is important statistics. They’re good for estimations. But we also need to bring in the qualitative insights.

      Nkiru: The single most instrumental fact on people not owning smartphones is actually the price of smartphones. Well, yes, there are obviously qualitative reasons why people don’t have, you know, different people — but the majority of people don’t own smartphones because they’re expensive. And that’s what’s cool about Android compared to iOS, which is that, you know, you can now get an Android phone for, like, $17. And I’m sure in the next year or two, there will be Android phones for less than $15. And I think the cheaper the phones get, the more people will have them.

      Sonal: Alan, do you wanna jump in here and maybe share a perspective on what you’re seeing from a more systemic level, working with governments across Africa? Especially because I think what Nanjira brought up about the last mile is a really, really interesting point. What are some of the obstacles that you’re seeing culturally as well as technically?

      Alan: I’ll just give you an example. In a South African township, in a place like Soshanguve or Mamelodi, 70% of people, if they want to apply for a job, they need to get in a taxi. They need to go to the local community center, they look on a board, they look for the jobs posted on the board, they write down the contact details for the job, they get on a taxi, they go to a printing shop, they print their CV, they get in a taxi, they go to the offices of the employer, and they hand in their CV, and then they wait for somebody to get back to them. And that’s, you know, between $4 and $10, kind of, cost and takes you a whole day. And that’s something that, you know, a lot of people in the world haven’t done for 20 years. So just to — you really have to go back to basics. You know, there’s no ubiquity of the internet for most people in Africa. 

      We’re now involved with the deployment of the largest public free Wi-Fi network on the continent. And, you know, we find there’s some interesting behavioral stuff. First thing is that there’s not a very big penetration of smart devices. So, Wi-Fi-enabled devices, maybe 50% of households in low-income communities — but there’s a big sharing behavior. So, a lot of people are sharing devices. So, it’s not so much about how many people have a device, it’s about how many households have a device. And if there’s between 5 and 10 people living in a household, you know, then your device penetration goes up quite a lot, because those 5 or 10 people are sharing one device to get on the web. Secondly, a lot of low-income communities who are not accustomed to the internet are still measuring data in minutes.

      So, if it takes you five minutes to download a video, they think it’s more expensive when it takes you one minute, even if the video happens to actually be more data, and you’re just on a faster network. So, there’s a lot of education that has to go around there. Old people — basically, anybody over there doesn’t really know what to do with the internet when you give it to them. So, it’s a bit like being an American in 1995. You need to have, like, a landing page from AOL, Yahoo saying, “This is the internet. Click here for, you know, something that’s useful.” So, you can’t give people a clean Google page. It’s absolutely meaningless for most of the communities we deal with. It’s like, “What is this? What are we gonna do with this?” And some of the communities aren’t literate either, which brings me to the content. And the content is — I mean, music videos are just a killer category, which is fun. That’s gonna make lots of money. 

      Christianity is a massive — faith-based content, particularly Christian faith, is massive, and European football. I mean, in South Africa, the Premier Soccer League is quite big. But just generally, English Premier League and European football, particularly clubs like Chelsea with lots of African players, it’s a massive content category. You don’t understand how many people are following that kind of stuff. And we see in our networks that Blackberry is a massive chunk of the market. I mean, I know in America, Blackberry is dead. You know, people laugh about Blackberry, but around here, it’s still pretty big, and Huawei — Huawei Android devices are pretty big. So, I think it’s a little bit different from what’s happening in the States.

      Sonal: That’s actually super helpful for fleshing out some of the statistics we’re talking about. Together you guys are, sort of, sharing both different ways of putting those numbers that we typically hear in context. I think what you pointed out about measuring things in terms of people versus devices in the household is really important — to talk about that penetration rate, and even the misconceptions people have around minutes versus data size is also really interesting. And then we also talk about some of the more qualitative behaviors that, you know, go with the power structures that you mentioned, Nanjira — where it’s, sort of, like, about, like, you know, is the woman the one who has a device? And she may be in the same household with, like, 10 other adults, but is her power and access equal to, say, the men in the house? I mean, it sounds like there’s a lot of different things to think about.

      Alan, I think it’s also really interesting that you talk about entertainment-based content being very popular, because you would almost think that with this being, sort of, this utility model that we’re all sort of arguing for, it’s interesting that the things that people find most popular are actually still down to entertainment, music, sports. For some people, they would consider faith an entertainment. I mean, it’s sort of, like, how people pass their free time in a lot of different regions. That’s actually kind of counterintuitive to me. And how does that play out with your observations, Nkiru, given that you run a music company?

      Africa’s growing middle class

      Nkiru: It’s a big continent, and there’s definitely a lot of poverty. But in saying that, there’s a growing middle class. And, you know, you can’t solve Africa’s problem or the world’s problem in one, you know, fell swoop. So, is the growing middle class big enough for us to, sort of, like, look at? I think so. I think that, you know, the whole picture of, you know — when you see on CNN when there’s, you know, black kids swatting flies, and sort of, that’s not — I mean, yes, there’s poverty, but there’s great stuff happening here as well. So, entertainment is huge because people are — people have, you know, more income. Well, maybe not in Nigeria. Right now, we have a recession. But people have more income and they want us — you know, they want to spend money, you know, doing chill, cool things. And, for example, going back to mobile phones, they’re aspirational. 

      So, you know, in the way that you want to, sort of, move to Lagos, because you’re in the town — so, you’re in the rural area and you want to move to Lagos. Things are aspirational here. People want to be cool. People want to associate. Being Nigerian is cool now. Nigerian music is cool compared to American music. You know, those kinds of things are things that we consider. So, there is a huge growing middle class, we think, and that’s why you see that. Well, some people are poor, and some people are poorer, and some people are less poor, and some people are really rich. And there’s I think — there’s a spectrum of people that are all-African, and we can, sort of, like, look at different perspectives rather than just focusing on all the bad stuff that goes on here, as everywhere else.

      Sonal: I think it’s interesting actually because you guys are painting a whole range, which is sort of the fact that the mobile phone can be everything from utility — which we’ve heard that common statistic, that more people have mobile phones than they do toilets, in a lot of developing countries. And then at the other end of the spectrum, you have this notion of the mobile phone as a very aspirational device, ecosystem, and all these things that come with it. I think the point you’re drawing, about focusing on the growing middle class, is incredibly important, because that is on a continuum. I mean, we can’t be focusing only on one extreme, the very rich or the very poor either. It’s certainly true in thinking about this.

      Nanjira: Yeah. I’d say another angle is also — let’s even imagine a utopia where every African has a mobile phone. Are they actually able to create? Are they able to code on that mobile phone and contribute to another <crosstalk>, or they’re just going to be consumers? And this is where we need to start asking these questions around what access to the internet, and how we’re facilitating — so that this fixation on a particular device does not, rather, inhibit us from seeing a bigger picture on how people could actually contribute and benefit from the global economy that’s also gonna be a very digital one.

      Sonal: There’s this famous quote about Steve Jobs. I don’t know if it’s true or not, if it’s just anecdote — saying that, you know, getting this criticism that the iPad was only intended to be, you know, for consumption, and him feeling very, you know, affected by that and trying to, you know, work on that, and clearly this huge ecosystem of apps have grown up around — since the launch of the iPad, where people can actually produce versus just consume on mobile devices. That said, we tend to take a very academic approach, I think, to that debate. It’s almost like frosting on the cake, versus what you’re describing is actually, again, incredibly important to the notion of inclusion, and being included in this larger global movement. Like, can you create code? Can you create art? Can you not just be a consumer who’s taking yet another technology from elsewhere, or even locally, and just providing money to it, versus actually being able to do something with it?

      Nkiru: So, when Nanjira is talking about, “Oh, can you code on your mobile phone? Can you do this?” I’m thinking, “I don’t even have water.” I need to have generally — I need to have, you know, like, lights. You know, I need to have power 24/7. So, it’s really, sort of, like — I don’t then say, “Oh, there’s no power here.” Because we have to produce our own power. Like, at Spinlet here, we really have to run a generator 24/7. You know what I mean? So, these are crazy things that happen here, but yet, you know, we’re still thinking about, you know, how we wanna code. So, I think it’s just an interesting place to be, where you can do both.

      Sonal: Yeah. No, I think it’s fascinating. It’s, like, collapsing Maslow’s hierarchy of needs. It’s sort of, like, saying — there’s, like, this funny diagram that periodically makes rounds on the internet where it’s, like, Maslow’s hierarchy of needs — where you have basic things for survival and at the very bottom, it’s Wi-Fi, exactly.

      Nkiru: Yeah. I mean, I must say, it’s really interesting to see how even in development considerations and development agendas, what we’re starting to see is whether it’s a zero-sum game. So, do we invest in water or internet? It’s starting to become a very short-sighted question. You know, does it mean, then, we should not focus — should we get everyone access to water so that we then get people internet? So the hierarchy of needs is being renegotiated, even from a state perspective or a donor perspective, you know. And each, you know, there’s so many nuances to each aspect, because either way, even if we waited and divested money into bringing in water first, we’re still going to be missing out for those who already have access or those who could have access. So, it’s a really interesting development question.

      Alan: So, the main thing is governance. You know, if you’re living in South Africa or Nigeria or Kenya or Zimbabwe, or any part of Africa — actually any part of the world, come to think of it — we’re all kind of complaining about the same thing. You know, our leaders aren’t doing a good job, etc. So, when it comes to votes, and deciding who gets them to power, we’re never gonna have better leadership if people don’t know what’s going on.

      Sonal: Oh, that’s a good point. So, like, sort of knowledge as a way to, sort of, break down some of that discussion.

      Alan: Yeah.

      The popularity of messaging

      Sonal: I wanna revisit a theme that you guys brought up earlier, which is, you know — you mentioned, Alan, that people get a little taken aback when they see a clean Google page, because they, sort of, need some instructions on how to navigate for first-time internet users what to do next. One of the interesting insights that I heard on our podcast about China and India is how people actually get taken aback with a very clean, uncluttered page, because they’re so used to having limited bandwidth that they want high information density. Like, they’d much rather have a design where there’s just a bunch of links for the sake of being parsimonious about that use of bandwidth. What are some of the interesting things happening on the design side of all these apps and things that you guys are seeing and using in your various regions, or that you’re working on?

      Alan: You know, we deal with some U.S. companies that try to come into, you know, to Africa. I find that maybe some “first world” kind of companies underestimate the consumer. You know, so they think something pretty B-grade, they can get away with it. But, you know, the internet is here. People can look around and see for themselves what’s good and what’s not. So, provided it’s not consuming too much data, I think user experience — consumers seem to be demanding as good of user experience as anywhere outside of Africa. But you just have to make it mobile-centric. You just have to make it mobile-centric. You know, there’s just — it’s absolutely irrelevant whether it looks nice on a laptop, you know, it’s just not relevant here.

      Nanjira: For us, one way we try to understand that — and also not homogenize users, end-users — is encourage, especially startups that we house here and other clients really, to consider user experience designing there, you know, in their iterations of either applications, mobile websites, or any offering they’re giving to the internet. Because, yeah, it’s usually assumed, you know, there’s this standard typical user who just needs to get this and that and that, but they’re very — they’re different nuances. So, if it’s a woman, again, who’s trying to use a mobile health app, there are gonna be things that she sees to have value, maybe at the landing page — just seeing all the different ways they could go to one place, other than just landing on a single page, as you mentioned. So, there’s so many nuances.

      Nkiru: You know, everyone’s talking about mobile phones and how people are using mobile phones, but we found that — so, we’ve had to recently redo our browser, you know, website, because we found that people who have advanced-feature phones actually, you know, prefer to sort of, like — or can only sort of access us via browser, as opposed to downloading an app. And, in any case, when you download an app, it sucks up your data. So that’s really — you know, it’s a very curious thing, where we are still doing a lot of browser work as opposed to just concentrating on apps, you know — going the app way because of the mobile phone filtration. So, we find that the browser is equally important, if not more important, because of the fact that data is so expensive. So, when you’re accessing the internet via a browser, you don’t have to download an app that could, sort of, chop up all your data.

      Alan: Yeah.

      Sonal: It also puts to rest some of the academic debates people have about whether mobile-centric isn’t just about, like, being on the mobile phone. You’re saying it’s down to the nuance of whether you’re designing for a browser versus the app itself.

      Nkiru: Exactly, yeah. Just design — mobile-friendly designs — as opposed to, you know, when, you know — the whole new concentration on apps. Like, they’re expensive to run. They chop up all your data, and then that means that you can’t actually — data is one of the biggest problems here. It’s expensive. So, people are counting every second they spend, you know, on the internet.

      Nanjira: And apps are not that popular, really, actually. A lot of studies we’ve done especially for, you know, applications for governance issues. They’re not popular. People are happy to either get their information, again, from the popular app. So if you go to Facebook, you’ll probably have links to this page that also has information, and that kind of thing. So, yeah, before you invest in an app, you have to really check whether it’s actually the appropriate technology to use for advancing your agenda.

      Sonal: Right. I would actually say, anecdotally, that’s also very true in India. I just came back from a trip a couple of weeks ago, and I noticed how very few apps anyone around me had on their phones. I mean, what I am curious about is whether you see people doing a lot more stuff over messaging, the way we’ve talked about, like, what’s happening with WeChat in China, or WhatsApp in India, and elsewhere. Do you have any perspectives on where messaging plays out into all this?

      Nkiru: I mean, WhatsApp is popular everywhere.

      Nanjira: WhatsApp is huge here, but, you know, it’s a millennial thing, isn’t it?

      Nkiru: Yeah.

      Nanjira: You know, millennials react the same way everywhere in the world, and so WhatsApp is really huge, too.

      Sonal: Oh, is it really a millennial thing? I was gonna say, because — I was gonna say, in India what shocked me when I went a few couple of years ago was that grandmothers and aunts were using WhatsApp, and that’s what was surprising to me.

      Nanjira: This is true. But then you’re, like, just generally all the apps. So there’s Snapchat now. There’s all kinds of new things that I can’t keep up with, and I keep getting told about them from, you know, my team. So WhatsApp is huge, huge, but other things are becoming equally huge. I think it’s a trending — you know, like, the trend — something is in, and then something is out, and then they like, “Have you heard of the new thing coming in?” So I think it’s general — worldwide things that are trending everywhere. But WhatsApp is huge because you can make free calls from WhatsApp, if you have internet, of course, but then that means that you’re then — if you’re, I think, maybe middle class where you have internet access, and then you can then make cheaper calls from WhatsApp compared to using the usual Telco line.

      Nkiru: You’re mentioning very U.S.-centric apps, and I actually wanted to direct this to Alan to talk a bit about Mxit and its heyday.

      Alan: Well, so, you know, I have the questionable honor of residing over probably the biggest tech failure in Africa.

      Nanjira: Oh, no.

      Nkiru: It was a success at some point.

      Nanjira: This isn’t what we want.

      Alan: But at the same time, it was probably the biggest <inaudible>, so there were some good things out of it.

      Sonal: What were some of the learnings?

      Alan: Mxit’s success came from the fact that it made it really, really cheap for people to text one another. That was it. So SMS or texting was — it was too expensive, and Mxit was first moving and built this massive network effect. So, it’s a killer app. You know, one of the lessons is, never ever go up against American companies. You’re dead. Silicon Valley, they’re gonna crush you. So, you know, as soon as the smartphone, kind of, wave started breaking, you know, WhatsApp just won the race. It has won the race. It will win the race. I like WeChat. I mean, I don’t think — likes for WeChat aren’t to be underestimated — but just aspirationally, in our markets, we can see that WhatsApp is winning. It’s at a youth level. It’s at a middle-age level, and it’s [at] an old-people level. So, it doesn’t matter what it is. And I’m quite excited, actually, to see WhatsApp opening itself up to be a bit of a platform, because if you can plug that community into whatever application you’ve got, you know, boom, you don’t have to reinvent the wheel. And everyone’s trying to reinvent the instant messaging wheel. So, I think that that ship has left, and WhatsApp has won the race.

      Sonal: Well, it’s interesting that you talk about it opening itself up into a platform, because that’s exactly right to see what comes next. But what I think is really fascinating is the use cases where I’ve seen abroad, where people are actually using it less for messaging, in a typical way, and, for example, they’re using it as a news source. They’re using it to vote in elections, in certain places, or to, like, do certain things. I’ve seen relatives use it exactly the way you would use a social network, but instead of using an actual social network, they’re just using messaging to do all those things. Like, share status updates, share photos, etc. How is the use of messaging — I guess I’m trying to get a little bit more understanding of how people are using some of the messaging apps in Africa in this context, and anywhere in Africa — not just all over — but in this context of messaging as a broader trend.

      Alan: In South Africa, we have a massive crime problem. So, security is a big concern for a lot of people in South Africa, and WhatsApp has become the de facto means of communities organizing against crime.

      Sonal: Oh, wow. 

      Alan: So, streets all have, you know — the block, or street, or whatever street has got, like, a WhatsApp group. Everyone participates in it, and anything that happens or any suspicious vehicles or anything, you know, people are always — the feedback group is WhatsApp. And it’s, like, far and away the most powerful tool for something like that. And the way I see this, kind of, evolving around here is, one of the new services we’re helping the government deploy is the equivalent of Uber, but for the police. So, if you see a crime in — especially in a community like Mamelodi or one of the townships, the street names aren’t very obvious. The addresses aren’t very obvious. But if you can use GPS to pin where you are, you can report it to your nearest police cars, and they can, kind of, respond without having to go through call centers and all the translations in between. You’ve solved a massive problem.

      Sonal: Wow.

      Alan: But chat is a massive component of that, you know. And you need the officers to be able to communicate with the citizen and vice versa, and just, kind of — just the more feedback there is, the faster people can respond and kind of get to the incident. And that’s where — I mean, chat is integral to everything that we do, that we see in the app space, and WhatsApp will — just, you know, if you can plug WhatsApp into it, you don’t have to reinvent the wheel.

      Nanjira: Perhaps another way of looking at WhatsApp and, you know, similar chats, like WhatsApp is that it’s come — for me, it’s become so everyday that I don’t even think about it. You know, it’s how people interact, it’s how you get even, you know, work conferences on WhatsApp. Everyone, as you say, uses WhatsApp, but what I wanted to sort of, like, maybe, sort of, like, take us to is — when Alan talked about when WhatsApp came to the market, and what it did to Mxit — and I thought that that’s interesting — when the Americans come in. So, for example, Apple Music, Netflix is coming in. Netflix is already here. Apple Music is here. And so, what’s that gonna do to the, you know, existing African SMEs in the market? I think that would be interesting for me to, sort of, hear everybody’s perspective as well.

      The importance of local advantage

      Sonal: I guess what I really wanna get at here, as well, to build on your question is, is there a local advantage, or does it not apply when a global player comes in?

      Alan: Do not ever take on America. So, that’s the golden rule. But there’s lots of things that in Silicon Valley, you know, people — some guy sitting in a cappuccino — having his cappuccino in San Francisco is just not gonna think about. You know, if you’ve got a problem that’s unique to Africa, and you’re trying to tackle that problem — and it must be unique to, not necessarily to Africa, but just not at all obvious in Silicon Valley — then you’ve got a shot. But, of course, if you get any kind of traction in any of the spaces, you’ve got to make sure that someone in the States ain’t gonna, like, just see your idea, copy it, and just come take you on. And the real capital advantage other than the, you know, the hugely talented, aggressive, you know, hard-working people. Just the capital advantage is so overwhelming. 

      So, in my opinion, you know, we invest in businesses only if there’s a network effect, and if someone’s already dominated the market. So, something like e-commerce — you know, Jack Ma, or Baidu, or Tencent — they all benefited from, you know, kind of, closed markets to the U.S., where they could build up the network effects for e-commerce or for social. And by the time they, kind of, opened it up to foreign competition, you know, it’s kind of — the land grab had happened, and you’ve got this massive moat protecting you from Silicon Valley. And there are some things like that in South Africa, and I know in some places, the rest of Africa — but there’s not a lot, and you really have to have this buyer-seller or friend-friend network in place before. Or know that you can get it before anyone in America gets wind of it.

      Sonal: Alan clearly has a very specific perspective. And do you guys agree, disagree?

      Nkiru: No, I think I generally agree but, of course, there’s — I mean, local advantage is, I mean, important and necessary, but the thing about local advantage is that local advantage can be bought. So, for example. So, I train my staff, and clearly I’ve spent so much time training them. And then, you know, whoever — the American person is coming in, and they have a lot more money, like, than I do. They have the capacity. They have tech capacity. They have, you know, capital capacity. And it doesn’t take, you know, a huge, you know — what do you call it, increase — to just make them jump. Am I angry about that? Maybe, when I’m really sort of, like, “Oh, my God. I spent all this time training these people.” But then, I also don’t wanna have people working at Spinlet, for example, where nobody wants to hire them. So, it’s like, you know, what came first, catch-22. But I think, you know, an Apple or an Amazon or — I think local advantage is very important, and I’ve talked about, you know, you can’t be an Africa expert just by spending two weeks here. But also you can’t compete with the American power of, you know, capital and, you know, tech power as well, so that’s how I see it. So, I’m with Alan on this one.

      Nanjira: I’ll give maybe two examples of where local advantage is giving a quick win for some startups here. One is very well-known. That was Ushahidi. I mean, it came up from a very contextual situation and occurrence here, but that was a specific use case. But for them to survive, they’ve also had to, you know, to embrace the Americans. So, Ushahidi has a team that works from the U.S., a team that works from Kenya, and a team that works from other countries — and that’s how they’ve been able to co-opt that expertise that’s ready in the market in the U.S., for instance, so that they’re not necessarily bought out. Another example that’s starting to come up now is the BRCK that’s being built as appropriate technology for facilitating Wi-Fi router access, right? The idea being that, you know, while I may buy a router for my house in the urban area in Nairobi, it may not necessarily be the most appropriate technology for beaming Wi-Fi access when I’m out in my grandmother’s village. You know, so it needs to be something rugged, something that could withstand, you know, falling and dropping and that kind of thing. Now, they have that local advantage by understanding that, but it also becomes a game of wits, in terms of — once you start gaining traction, as was mentioned, once you start gaining attention, and other people start to see what you’re on to, how do you make sure they don’t beat you to market in something that’s faster, cheaper — and there’s the second-to-market advantage. So, it becomes a game of smarts here, and it’s one we’re learning by — you know, it’s baptism by fire, if ever there was a case.

      Sonal: Right. Well, it’s fascinating, too, because what you’re describing is competition everywhere, but there’s also a factor here where there’s a power differential between, as you mentioned, access to capital, access to — you know, it could be like Chinese investors coming into India, or Jeff Bezos going to India with Amazon, or it could be — right. I mean, no one can compete with that, sort of, deep pockets at a certain level. But at the same time, we do see success cases where they compete in different ways.

      Nkiru: The one thing I can say about a local advantage, where it really is an advantage, is when, like, an international is coming in to — you know, is trying to establish footprint on the continent, and the general mistake that, you know, internationals seem to make is that they assume that by hiring someone out of, you know, business school from somewhere — I don’t know — they have — you know, they can do it. And that’s where you need the local — that’s where the local advantage will always, you know, sort of overpower the — you know, the — because you cannot come to, you know, Lagos, if you’ve just, you know, been here one week, and you cannot go to Kenya, and, you know, set up a company without having, you know, the local insight, the local, you know, local knowledge. Just, sort, of how to navigate the terrain. And so, that’s where local advantage would always, you know, beat, you know, the clout and the power. And so, I think it’s just finding a way to do things in collaboration.

      Doing business in Africa

      Sonal: Reflecting on this notion of competing with business around the world, and locally, any final reflections on things that people should know who are trying to build businesses in Africa?

      Alan: Yeah, I once read a blog that Silicon Valley, you know, gives you 16 metrics to chase for a startup. And a lot of people in South Africa and Africa, and the rest of the world, read those blogs and think, “That’s the holy grail.” But there’s only one metric, and that’s cashflow. Advertising is not a business model. “One day we’ll sell to Google” is not a business model. You really need to be, kind of, thinking like a traditional business — like, how are you gonna make money pretty soon. And if you’re not, well, then you’re really up against some guy who’s funded out of Silicon Valley.

      Sonal: So, one final question that’s come up a few times is — thoughts on, you know, technology and how women are using it at all ages, in various regions of Africa. You’ve brought up the notion of women doing things a little differently a couple of times. I wanted to make sure you guys could share some thoughts on that.

      Nanjira: Sure. You know, the World Development Report by the World Bank just came out and reconfirmed the notion that, yes — much as there are many women who are participating online, the numbers still lag behind in comparison to men. And that could be seen, you know, as an indicator of existing inequality, so to speak, but the internet and access to it — for those who do get access to it, and who are able to bypass certain barriers that are socially, culturally, you know, income-wise, who get to participate — we are seeing such opportunities to raise agency. To have agency to, you know, speak up about issues, to address and to really mainstream the idea of gender in very various considerations.

      So, for instance, you know, it’s no longer sufficient to just have numbers of women on a panel. It also goes now to, “We will criticize whether they actually get to be asked questions based on their expertise, and not just because they’re filling a diversity quota.” And so, you’re going to see these kinds of conversations taking place online. There’s so many campaigns we’re seeing that are helping mainstream the idea — taking on, you know, I would say misogynists who are both men and women. We’re seeing fantastic campaigns coming up about that. Sensitization on these issues in a way that’s not just preachy and traditional NGO-y, where it’s just driven by citizens who are engaging on platforms that facilitate free expression. So this is fantastic. That said, there’s still women who are keeping off because they’re attacked, and there’s a lot of digital safety considerations to have. And this, you know, it’s not to make women seem, like, some little eggs in a sense, it’s just — these are just indicators of existing patriarchal structures and so it’s really — the internet and access to it is really one of those things that could either perpetuate this patriarchy, or start deconstructing it as we all get connected.

      Sonal: Or probably, more likely both, actually.

      Nanjira: Yeah, or both. I mean, society does exist in contradictions. And in fact, as a parting shot, I also wanted to add that there’s a fact that we need to have more research supported to really unearth these qualitative insights, that then complement these numbers that we’re getting. I mean, I don’t know whether to be offended or amused by the statistic, you know, that they’re more mobile phones than toilets. I get that statistic, but then I’m, like, “To what end are we — what are we saying?” You know?

      Sonal: Right. What does that mean exactly? Right. That’s a good point. We use it actually. I’m gonna confess, we use that statistic ourselves. I’ve linked to that or quoted that in various pieces, and you’re right to call BS on it a little bit. I mean, not BS necessarily, but to say, like, “What does that even mean?”

      Nanjira: What does that actually mean?

      Nkiru: That is actually BS.

      Nanjira: It’s a statistic to be used <crosstalk> for a U.S. state, so there are more mobile phones than there are toilets — does that apply across the metric, if you’re using it as a comparative? Or is it just for Africa and the developing world? So, there’s a need for more nuanced research to strengthen and, sort of, flesh out those numbers being thrown out. So, let’s not just quantify Africa. Let’s bring other qualitative insights that bring out its diversity, and also make for wise investment decisions, actually.

      Nkiru: I wanted to talk about like, you know, the idea of women in technology and how it’s been niggling at me for a while now. And I think that the phrase, in itself — instead of creating or, sort of, incubating, you know, a whole group of women who are get — then gonna have more low self-esteem, because or inferiority complex, because we then assume that you have to be great at math to do — to work in a tech company, for example. And, you know, and I feel like when we, sort of, create these things that will help women — or we need to have women do more math — these are, like, 9-year-olds we’re talking about, and 12-year-olds, and 14-year-olds. There’s a whole group of 25-year-olds and 22-year-olds who are long out of school, who don’t need to be told that. They just need to be told that they need to excel in whatever they’re doing, because they can do anything. And so, I’m sort of worried about the terminology that’s suddenly creeping up in the past two years about, oh, how women need to be encouraged to be, you know, in STEM and all that stuff. So, I hope that we can talk about these things more frequently and sort of, like, put light to them.

      Sonal: Well, I actually agree with you personally, as well. I think that that discourse is important, but also troubling when it puts this hierarchy of skills that’s, sort of, not honoring some of the more nuanced qualitative sciences as well. Which is a meta-theme in this conversation as well. Alan, Nkiru, and Nanjira, I just wanna say thank you again for taking the time. This has been an incredible conversation. We’re gonna continue it, I’m sure, in the near future, and the first of many, and thank you.

      Nanjira: Thank you.

      Nkiru: Fabulous. Thank you so much.

      Alan: Thanks very much.

      • Nkiru Balonwu

      • Alan Knott-Craig

      • Nanjira Sambuli

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Artificial Intelligence and the ‘Space of Possible Minds’

      Azeem Azhar, Murray Shanahan, Tom Standage, and Sonal Chokshi

      What is AI or artificial intelligence but the ‘space of possible minds’, argues Murray Shanahan, scientific advisor on the movie Ex Machina and Professor of Cognitive Robotics at Imperial College London.

      In this special episode of the a16z Podcast brought to you on the ground from London, Shanahan — along with journalist-turned-entrepreneur Azeem Azhar (who also curates The Exponential View newsletter on AI and more) and The Economist Deputy Editor Tom Standage (the author of several tech history books) — we discuss the past, present, and future of AI … as well as how it fits (or doesn’t fit) with machine learning and deep learning.

      But where are we now in the AI evolution? What players do we think will lead, if not win, the current race? And how should we think about issues such as ethics and automation of jobs without descending into obvious extremes? All this and more, including a surprise easter egg in Ex Machina shared by Shanahan, whose work influenced the movie.

      Show Notes

      • Distinguishing various type of AI and how they learn [0:57]
      • Ethical concerns [11:13], and a discussion of how AI may develop going forward [22:02]
      • Will academia or business succeed in developing AI [30:15], and how might it affect jobs? [34:51]
      • Will AI ever become conscious? [36:18]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast,” I’m Sonal. And, today, we have another episode of the “a16z Podcast” on the road, a special edition coming from the UK. We’re in the heart of London right now. I’m here with Murray Shanahan, who is a professor of cognitive robotics at Imperial College in London, and he also consulted on the movie “Ex Machina.” And so, if you didn’t like the way that movie turned out — we don’t wanna put any spoiler alerts — you can blame him. And then I’m here with Tom Standage, who’s the deputy editor at “The Economist” and also the author of a few books.

      Tom: Oh, yeah, six books. The most recent one was “Writing on the Wall,” which was a history of social media going back to the Romans, and probably the best-known one in this context is “The Victorian Internet,” which is about telegraph networks in the 19th century being like the internet.

      Sonal: That’s great. And I’m here with Azeem Azhar, who publishes an incredibly interesting and compelling newsletter that I’m subscribed to — the “Exponential View.” He used to be at “The Guardian” and “The Economist,” and then most recently founded and sold a company that used machine learning heavily. So, welcome, everyone.

      Murray: Thank you.

      Azeem: Hello.

      Types of AI and learning techniques

      Sonal: So, today, we’re gonna talk about a very grandiose theme, which is AI — artificial intelligence — and just, sort of, its impact and movements. This is really meant to be a conversation between the three of you guys, but, Murray, just to kick things off — like, you consulted in the movie “Ex Machina.” Like, what was that like?

      Murray: Oh, it was tremendous fun, actually. So, I got an email out of the blue from Alex Garland, famous author — so, that was very exciting to get this email. And the email said, “Oh, my name’s Alex Garland. I’ve written a few books and stuff. And I read your book on consciousness, ‘Embodiment and the Inner Life,’ and I’m working on a film about artificial intelligence and consciousness. And would you like to, kind of, get together and talk about it?” So, of course I jumped at the chance, and we met and had lunch. And I read through the script, and he wanted a bit of feedback on the script, as well — where it hung together from the standpoint of somebody working in the field. And then we met up several times while the movie was being filmed, and I have a little Easter egg in the film.

      Sonal: Oh, you do? What was your Easter egg? I’ve seen that movie three times in the theater, so I will remember it, I bet.

      Murray: Oh, fantastic. So, there’s a point in the film where Caleb is typing into a screen to try and crack the security, and then some code flashes up on the screen at that point, and that code was actually written by me.

      Sonal: Oh, yay.

      Tom: So, it’s real code, it’s not the usual rubbish code.

      Murray: And it just, sort of, flashes up. But what it actually does, if you actually type it into a Python interpreter, it will print out “ISBN equals,” and the ISBN of my book.

      Sonal: Oh, that’s so great.

      Tom: Oh, it’s Python as well? I’m even more thrilled.

      Sonal: I think it’s fascinating that you say that the part that you didn’t go into detail about is the name of the second part of the title of your book, embodied consciousness?

      Murray: Yeah, there’s a long subtitle which is “Cognition and Consciousness in the Space of Possible Minds.” Now, I very much like that phrase, the space of possible minds. I think if you were to kind of pin me down on what I think is my most fundamental, deepest interest, it’s this idea that what constitutes possible minds is much larger than just humans, or even the animals that we find on this earth, but also encompasses the AI that we might create in the future, either robots or disembodied AI.

      Tom: So, it’s a Hamiltonian space of possible minds? That’s beautiful.

      Murray: Yeah, a huge kind of space of possibilities.

      Azeem: I mean, it’s a really interesting idea, and it’s something that comes across in a couple of your other books as well, which is this notion that we think of intelligence as — quite often the artificial intelligence — that plastic white mask that you see on the cover of many, many a film or book cover. But, of course, as we start to develop these new AI systems, they might take very, very different shape. They may be embodied in different ways, or they may be networked intelligence. So, one of the areas I think is interesting is what’s happening with Tesla, and the Tesla cars that learn from the road — but they all learn from each other. Now, where is that intelligence located, and what would it look like, and where will it sit in your space of possible minds?

      Murray: Yeah. Absolutely. It’s a completely distributed intelligence, and it’s not embodied in quite the sense of — of course, a car is a kind of robot, in a way, if it’s a self-driving car, but it’s not really an embodied intelligence. It’s sort of disseminated or distributed throughout the internet, and it’s a kind of presence. So, I can imagine that within the future, rather than the AI necessarily being the stereotype of a robot standing in front of us, it’s going to be something that, sort of, is hidden away on the internet and is a kind of ambient presence that goes with us wherever we go.

      Tom: Well, that’s another sci-fi stereotype there, isn’t it? That’s the UNIVAC or the Star Trek computer. But, as I understand it, your work starts with the presumption that embodiment is a crucial aspect of understanding intelligence, which is why you’re interested in both the robotic side and the intelligence side.

      Murray: So, certainly, I would’ve taken a stance, you know — if you’d asked me 10 years ago — that cognition and intelligence is inherently embodied. Because what our brains are really for is to help us get around in this world of three-dimensional space and complex objects that move around in that three-dimensional space, and that everything else about our intelligence — our language, our problem-solving ability — is built on top of that. Now, I’m not totally sure that it’s not possible to build AI that is kind of disembodied. Maybe — in my latest book, I use the phrase vicarious embodiment. Or, I should say vicarious embodiment, for a U.S. audience.

      Tom: So, it can kind of embody itself temporarily in a thing and then go somewhere else?

      Murray: Oh, well, that’s another thing, you can have sort of avatars. But what I mean by vicarious embodiment is that it uses the embodiment of others to gather data. For example, the enormous repository of videos there are on the internet. There are zillions of videos of people picking up objects, and putting things down, or moving around in the world. And so, potentially, it can learn in that vicarious way everything that it needed to learn by being actually embodied in itself.

      Tom: And this goes right back to the neurological basis, I think, of some of your — because you started off doing symbolic AI and then moved over as, kind of, the whole field has, to more of this neurological approach.

      Sonal: And by neurological approach, Tom, you mean more like in a deep learning sense?

      Tom: Well, exactly. As I understand it, part of your approach there was the idea that the brain itself can rehearse motor-neural, sort of, combinations, and that’s how we, kind of, predict how the world will behave. We kind of say, “What would happen if I did this,” which is very much like what the DeepMind AI is doing when it plays Breakout or whatever — those, kind of, Deep Q networks, which is all about feedback based on predicted actions and remembering how things worked out in the past.

      Murray: Certainly. I’ve always thought that this idea of inner rehearsal is very, very important — our ability to imagine different possibilities and…

      Tom: So, watching YouTube videos of people doing things can function as inner rehearsal, I think.

      Murray: Or, if you have a system that can learn from that, the sort of dynamics of the world and the statistics of actions and their effects and so on — then it can use that — so, it sort of builds a model of how the world works, and then it can use that model to construct imaginary scenarios and rehearse imaginary scenarios. Actually, just going back very quickly to the DeepMind, DQN. So, in fact, for the bit of work that they actually published — I think one of its shortcomings, actually, is that, in fact, although it has done all of that learning about what the right actions are doing in the right circumstance is, it doesn’t actually do inner rehearsal. It doesn’t actually work through scenarios. It just…

      Tom: Oh, it’s just remembering how things worked out in the past?

      Murray: Yeah.

      Sonal: Murray, actually, what exactly then is inner rehearsal? Because I think I’m actually confused. We’re describing three different things. There’s sort of a predictive aspect, there’s sort of this decision-making framework, and then there’s also, sort of, something that reacts to the world in a dynamic environment that’s constantly changing, and reacting to that information in a very proactive and intentional way. Those are all different qualities. So, what is inner rehearsal exactly?

      Murray: So, I think that the, sort of, architecture of intelligence is putting all of those aspects together, really. So, inner rehearsal is when we close our eyes — of course, we’re not really necessarily gonna close our eyes, especially if we’re on the underground and the — but it’s when we close our eyes and imagine going through some particular scenario. Imagine doing an action and inwardly realizing that this will be a good, or this would have a bad outcome.

      Sonal: It’s like a planning scenario. In model-based reasoning, it’s like planning…

      Murray: Sort of planning, yeah. It’s model-based reasoning, yeah.

      Tom: Some of the same bits of our brains light up. And if I imagine punching you, then, actually, the parts of my brain that will be involved in punching you are partly — fortunately, they’re not actually…

      Sonal: Punching you. They’re just envisioning that similar — right.

      Tom: But the point is there’s more to it than just, sort of, thinking of the scenario. In some sense, the brain does rehearse the scenario in other ways, doesn’t it?

      Murray: Yeah. There’s quite a bit of evidence. The way the brain does it, as you say, is to actually use the very same bits of neurological apparatus that it uses to do things for real.

      Sonal: So, the planning is almost interchangeable?

      Murray: Yeah, it’s just kind of turning off the output and the input.

      Tom: And I remember seeing a video of a cat, and the cat’s acting out its dreams because it had some part of its brain basically modified, so that the part that normally suppresses the intention to act things out that you’re rehearsing actually was taken away. And so, the cat was imagining swiping mice and this sort of thing while being asleep.

      Sonal: So, that’s actually kind of fascinating, because it’s the reversal of how I’ve always thought of the human brain, which is — you’re basically saying, almost, that there’s always a bunch of scenarios and actions that can play out at any given moment in the brain, and that we’re actually already acting on, in essence, by the neurological impulses that are being fired in the brain. But in reality, what’s holding it back is some kind of control, that’s stopping something from happening, as opposed to saying, “I’m gonna do X, Y, or Z,” and then acting on something intentionally. So, it’s more of a negative space thing than a positive thing.

      Murray: Yeah, or a kind of veto mechanism.

      Sonal: Right.

      Murray: Yeah. Oh, in fact, I think you’ve actually proposed [that] there two very good rival hypotheses for what’s going on. And I wouldn’t want to venture what I think is the answer there, and it’s the kind of thing that neuroscientists study.

      Azeem: But it doesn’t feel like current AI — certainly, the stuff that’s implemented commercially, or even that’s published at a research level — is really bridging into this area that we’re talking about, these rehearsal mechanisms, for example.

      Murray: Yeah. I think it’s actually one of the, potentially, hot topics to incorporate into machine learning in the not-too-distant future. So, one of the fundamental techniques in DeepMind’s work is reinforcement learning.

      Sonal: Which is also very popular in developmental psychology.

      Murray: It has its origins, really, in things like classical conditioning.

      Sonal: That’s right, Pavlovian classic, bells, signals.

      Murray: So, within the field of reinforcement learning, there’s a whole little subfield called model-based reinforcement learning, which is all about trying to do it by building a model, which you can then potentially use in this rehearsal sort of way. But although Rich Sutton, who is the sort of father of reinforcement learning — in his book, way back in 1997, he proposed his architectures in which these things are blended together very nicely. But I don’t think anybody’s really built that in a very satisfactory way quite yet.

      Ethical concerns

      Sonal: So, just to help us come along with this — concretely, where are we right now in this evolution? And there’s schools of thoughts that can disagree with this, but just to simplify things — machine learning, deep learning as a deeper evolution of machine learning, and then sort of like a full AI on a continuum. Is that sort of a fair way to start looking at it? And where do we kind of stand on that continuum?

      Azeem: So, I have a model which says that, you know, AI and machine learning are really quite distinct things. You know, AI is all about building systems that can, in some way, replicate human intelligence or explore the spaces of possible minds, in Murray’s phrase — whereas machine learning is a very specific technique about building a system that can make predictions and learn from the data itself. So, there are AI efforts that have no machine learning in them. I mean, COIC, C-O-I-C, is a great example. You know, you try to catalog all of the knowledge in the world. I think, you know, it’s the mindset of the market to combine the two, because it might give something more attention.

      Murray: Yeah. I mean, I very much agree with that. I see machine learning as a kind of subfield of artificial intelligence, and it’s a subfield that’s had a tremendous amount of success in recent years, and is gonna go very, very far. But, ultimately, the machine learning components have to be embedded in a larger architecture, as indeed they already are, you know, in some ways, in things like DeepMind’s…

      Tom: We’ve had this sort of thing before though in the history of AI, haven’t we, where particular approaches have been flavor of the month, and you’ve got the expert systems for one. I mean, there was the early neural nets, which were much smaller neural nets, and now bigger neural nets and deep learning based on that, and, sort of, these systems that are sort of self-guided learning seem to be flavor of the month. But given that you’ve been in the field so long, do you see this as, you know, something that’s likely to run its course, and then will move on to something else?

      Azeem: Is it the end of history?

      Murray: So, I think there might be something special this time, and one of the indicators of that is the fact that there’s so much commercial and industrial interest in AI and in machine learning.

      Tom: But that reflects the fact that it’s been making a lot more progress than any of those previous attempts.

      Murray: Exactly, yeah.

      Tom: Isn’t there a problem, though, with the expert-based systems, that you could ask them why they reach particular conclusions? And with a self-driving car based on an expert system and, you know, it decides — and the classic, you know, trolleyology dilemma of does it, you know, run over the…

      Sonal: Oh, the school bus with the children?

      Tom: Yeah, exactly. All of those sorts of things, I mean, which I think are very interesting, because even now, you have sort of implicit ethical standards in automatic braking systems. You know, is it small enough — if it’s that small, it’s probably a dog, if it’s this big, it’s probably a child.

      Azeem: So, I think the trolley problem is definitely worth looking at and talking about, because we, as humans, don’t even agree on what the correct outcome should be.

      Tom: So, if we’re thinking about the trolley problem, and one of these scenarios comes to pass, with an expert system, you know, rule-based system, you could say, “Why did you do this,” and the system will be able to say, “Well, basically, this rule followed,” and da, da, da. And with these more elaborate systems, where it’s more like gardening than engineering the way we built them — it’s much, much harder to get any of that kind of thing out of them. And it makes them much more capable, but isn’t that gonna be problematic, potentially?

      Azeem: So, I think there are still objectives that we understand, right? So, the way that you build a system that predicts using machine learning is very utilitarian, right? You say there’s some cost function you wanna minimize, there’s some objective function we want to target, and then you train it. And you don’t really worry about the reasoning, because the ends, in a way, justify the means.

      Tom: But the ends are gonna vary. I think we’re gonna see, you know, get into a car and you can, like, adjust the ethics dial.

      Azeem: Right.

      Tom: Because that recent research that suggested that people are totally fine about cars making utilitarian decisions as long as it’s not them that, you know, is in there.

      Azeem: But in a way, we’ve lived in this world for a long time, we just haven’t had to ask the difficult questions.

      Tom: Yeah.

      Azeem: So, any time you pick up the phone, in the UK, it’s to your utilities provider, in the U.S., I understand it’s the Comcast clerk and customer service, you’re forced through an algorithm. You’re forced through a non-expert expert system, where the human at the other end has no discretion and has to just ply their way through a script, and we know how frustrating it is to live in that world.

      Sonal: Very much so.

      Azeem: Now, as we embed these AI-based systems, or machine learning-based systems, into our everyday lives, we’re gonna face exactly the same issues — which is, my car didn’t do what I wanted it to do, my toaster didn’t do what I wanted it to do — and I have no way of changing that. And so, this question about where is the utility function and what is the tradeoff has been designed in systems for 30, 50, 100 years or more.

      Tom: It’s just becoming explicit now.

      Azeem: It’s becoming much more explicit because it’s happening everywhere.

      Murray: Also, I think there’s a big issue with these kinds of systems, which may work just the way we want them to work statistically. So, if you’re a company, then you know that it makes the right decision for 99% of the people who phoned up.

      Sonal: Right, sort of an actuarial analysis.

      Murray: If you are the 1% person who’s phoned up and got a decision which is not one that you like, then…

      Tom: So, you don’t wanna be told, “Well, it was right statistically.”

      Murray: Or, just “computer says no,” you know? You want to have reasons. Or more seriously, if you’re in government and you’re making some big decision about something, or in a company and making a big decision about something, you don’t want the computer to just say, “Just trust me. It’s statistics, man.”

      Tom: Yeah.

      Murray: You know, you want a chain of reasoning.

      Tom: That brings up another aspect of this, which I find quite amusing, which is that there are quite a lot of sci-fi future — Sir Iain Banks’s future, and the Star Trek future — where you basically have a post-capitalist society, because you can have a perfect planned economy, because an AI can plan the economy perfectly. But, you know, there is a question of how plausible that is. But I wonder, you know, the extent to which you think AIs will start to be used in policy-making and those sorts of decisions.

      Murray: Well, I suspect that they will be, and I think that’s why, in fact, this whole question that you’re raising — of trying to make the decision-making process more transparent, even though it’s based on statistics and so on — I think that’s a very important research area.

      Azeem: And I think I would separate out the two areas of transparency. So, one is the black box nature, right? Can we look inside the box and see why it got to the conclusion it got to? The other side part that’s important is to actually say, “This is the conclusion we were aiming for.” And within policymaking, what becomes interesting, then, is forcing policymakers to go off and say, “That extra million pounds we could’ve put into heart research, we didn’t, even though it cost four lives. And we put it in something else because we needed to.”

      Sonal: The kind of analysis we’re talking about — this actuarial analysis — we’re doing it every day already with insurance, which is just distributed risk.

      Azeem: So, we don’t mind if humans do it, but we might mind if machines do it.

      Murray: I think that’s the big issue is — will we be happy to hand over those decision-making processes to machines. Even if they made exactly the same decisions on exactly the same basis, you know, will society accept that being done in this automated way?.

      Azeem: But, in a sense, we already have. It’s called Excel. It’s not even so much that we trust whether it works as a human works. What we’re doing is using Excel — we’re allowing ourselves to manipulate much larger data sets than we could’ve done just with pen and paper.

      Sonal: Exactly. We’re sort of organizing the cells in our mind into cells in a spreadsheet.

      Azeem: Into cells in a spreadsheet. And instead of having 100 data samples, you just look at 16 million and whatever, you know, that Excel can handle. So, we’ve already started to explore the space of decisions using these tools, right, to extend human reach.

      Sonal: There’s some of the examples that kind of approximate where we can go. Because the examples that come to mind — I think historically of Doug Engelbart’s notion of augmented cognition, augmented intelligence. And then I’m even thinking of current examples, like Stephen Hawking. Helene Mialet wrote a beautiful book called “Hawking Incorporated” about how he’s essentially a collective. I mean, I don’t agree with this turn of phrase, but describing him almost as a brain in a vat, surrounded by a group of people who are anticipating his every need. And it’s not just, like, Obama’s crew who’s helping him get elected and his support team. It’s actually people who understand him so well that they know exactly how to help him interpret information.

      Tom: Sort of like a group organism. In your space of possible minds, we have a whole bunch of minds that we could be, you know — and some people are trying to figure out already — which are animals, and then you’ve got the sort of social animals, the group minds there. And this, kind of, brings us to another ethical question from the previous one we were talking about which is, you know, the whole question of the evidence that octopuses are very — octopodes, we should say — are extremely intelligent, you know, has made some people change their mind about whether they want to eat octopus.

      Azeem: Yeah, so I don’t eat octopus anymore.

      Tom: You don’t eat octopus anymore. So, really…

      Sonal: I’m vegetarian, so I don’t eat anything that…

      Azeem: And as of today, I don’t eat crab either.

      Tom: Anyway, so there’s the point of the extent to which a creature with a mind that we recognize is cleverer than we thought, whether it’s right for us to boss it around. But we’re gonna get this with AIs as well, aren’t we? Because the usual scenario people worry about is we are enslaved by the AIs, but I’m much more interested in the opposite scenario. Which is, if the AIs are smart enough to be useful, they will demand personhood and rights. At which point, we will be enslaving them.

      Azeem: So, let me give you a practical example of that. There’s an AI assistant called Amy, which allows you to schedule calendar requests. And so, you know, I’ll send an email to you, Murray, and say, “I would like to meet you,” CC Amy. And then Amy will have a natural language conversation with you, and you think you’re dealing with my assistant. One of the things that I found was, I started to treat her very nicely. Because the way she’s been designed as a product, from a product manager perspective, is very thoughtful.

      Tom: So, you didn’t say, “Organize lunch, slave.”

      Azeem: Exactly. I didn’t do that, and I was quite nice to her. And then I had a couple of people who are incredibly busy write very long emails to her saying, “I could try this, or I could try this. If it’s not convenient, I could do this,” and I thought, “This is just not right. There is a misrepresentation on my part.” So, I then started to create a slightly apartheid system with Amy, which is — if you’re very important, and, Murray, you fell into that category — you’ll get an email directly from me, and other people will get an Amy invite. And it does start to raise some of the issues that are very present-day, right? They’re very present-day, because right now we have these systems.

      So, I think one of the ethical considerations is, we need to think about our own attention as individuals and as people here. And as we start to interface with systems that are trying to be a bit like the Turk — the chess-playing device that pretended to be a human — we’re giving attention to something that can’t appreciate the fact that we’re giving it attention. And so, I’m now using a bit of computer code to impose a cost on you.

      Tom: Well, actually, it’s like when I speak to an automated voice response system, you know, I speak in a much more precise way when it says, “Read out your policy number.” I know I’ve got to help the algorithm. I’m not…

      Sonal: Right. We’re shaping our behaviors to, sort of, adapt to it.

      Tom: Exactly. So, we already do it. When we type questions into Google, we miss out the stop words, and we know that we’re just basically helping the algorithm.

      Sonal: We don’t ask questions anymore. We peck things out in keywords.

      How AI may develop

      Murray: I think Google will expect us to do that less and less as time goes by and expect the interactions to be more and more in natural language. So, I think between the two of you, you’ve raised the two, kind of, opposing sides of this deeply important ethical question about the relationship between consciousness and intelligence, and consciousness and artificial intelligence. Because, on the one hand, there’s the prospect of us failing to treat as conscious something that really is — that’s very intelligent — and that raises an ethical issue for how we treat them. Then, on the other side of the coin, there’s the possibility of us inappropriately treating as conscious something that is not conscious and is, you know, perhaps not as intelligent. So, both of those things are possible. We can go wrong in both of those ways.

      And I think this is really one of the big questions we have to think about here is — and I think the first really important point to be made is that there’s a difference between consciousness and intelligence. And just because something is intelligent doesn’t necessarily mean that it’s conscious, in a sense of “capable of suffering.” And just because something is capable of suffering and conscious doesn’t necessarily mean that it’s terribly bright. So, we have to separate out those two things for a start before we kind of have this conversation.

      Sonal: That’s a great point.

      Azeem: And I think the thing we seem to care about is consciousness, from an ethical perspective, because we care a lot about the 28-week preterm baby, which is not very intelligent to…

      Tom: And we care about dogs and cats as well.

      Sonal: So, wait, where are we then when people have expressed fears? Because one of the things I think has compelled me to invite all three of you in this discussion is, none of you fall into one of these extremes of, like, completely, you know, cheerleading — like, “The future is dead,” and, you know, “We’re gonna be attacked and taken over.” Or the other extreme, which is, sort of, dismissive, like, “This will never happen, ever.” Where are we?

      Tom: We’re all in the sensible middle, aren’t we?

      Murray: I guess you’re asking where are we, you know, historically speaking now, right?

      Sonal: In this evolution and this moment.

      Murray: And I think the answer is we just don’t know. But, again, there’s a very, very important distinction to be met. This is the trouble with academics. We just wanna make distinctions, you know?

      Azeem: Distinctions.

      Tom: Journalists want to make generalizations.

      Murray: Yeah, they all can be important, and this is a case where it’s really important to distinguish between the short-term specialist AI — the kind of tools and techniques that are becoming very, very useful and very economically significant — and general intelligence — artificial general intelligence, or human-level AI. And we really don’t know how to make that yet, and we don’t know when we’re gonna know how to make that.

      Tom: You don’t sound like a believer in the, kind of, takeoff theory — that, you know, the AI is able to develop a better AI in, you know, less time, and so you get this sort of runaway. And I think that’s a very unconvincing argument. It assumes all sorts of things about how things scale.

      Azeem: So, I think the takeoff argument — it has a sense of plausibility. It’s the timing that’s the issue. So, I can’t deny the possibility that we could build systems that could program better systems, and that could start program better systems.

      Tom: But the point is that a system that’s twice as good, if it’s, say, you know, an order — it might scale non-linearly. So, it might be 256 times harder to build a system that’s twice as good. And so, every incremental improvement is going to take longer, and it’s going to take a lot longer. And improvements in other areas, like Moore’s law and so on, again, are not fast enough to allow each incremental generation of better intelligence to arrive sooner than the previous one. So, there’s a simple scaling argument that this need not be linear.

      Sonal: There’s also a classic complexity brake argument. I mean, there are so many different arguments.

      Azeem: There are lots. And, you know, as we start to peel apart the brain and our understanding of the neurological bases for how, kind of, cognition functions work, we learn more and more and we see more and more complexity as we dig into it. So, in a sense, it’s a case of, “we don’t know what we don’t know.” But we’ve been here before, before we’d understood this idea of there being a magnetic field, and needing to, you know, represent physical quantities with tensors, rather than with scalars or vectors. We didn’t see magnetic fields. We didn’t understand them. We didn’t have mechanisms for manipulating them, because we couldn’t measure them, and, therefore, we couldn’t affect them.

      And there would have been this whole set of physical crystals and rocks that were useless, because we didn’t know that they had these magnetic properties, and we didn’t know we could use them. Silicon dioxide being a great example — totally useless in the 17th century, quite useful now. And so, at some point, we might say that the reason we think this looks very hard, or it’s not possible, is because we’re actually just not seeing these physical quantities. When we touch on this idea of consciousness, you know, there is this idea of integrated information theory, which is this theory that, you know, consciousness is actually an emergent property of the way in which systems integrate information, and it’s almost a physical property that we can measure.

      Tom: Yeah, or we could be, like, I suppose, like Babbage saying, “I can’t imagine how you could ever build a general-purpose system using this architecture,” because he can’t imagine a non-mechanical architecture for computing.

      Murray: Right.

      Sonal: Murray, where do you fall in this singularity debate? And you’re not allowed to make any distinctions.

      Murray: Well, without making any distinctions, I’m still gonna be boringly academic, because I wanna remain kind of neutral — because I think we just don’t know. I think these arguments in terms of recursive self-improvement — the idea that if you did build human-level AI, then it could self-improve — I think there’s a case to be answered there. I think it’s a very good argument, and, certainly, I do think that if we do build human-level AI, then that human-level AI will be able to improve itself. But I kind of agree with Tom’s argument, that it doesn’t necessarily entail it’s gonna be exponential. <crosstalk>

      Sonal: Right. So, actually, to pause there for a moment, you started off very early on talking about some of the drivers for why you’re excited about this time — why this time might be different. What are some of those more specifically? Like, Moore’s law we’ve talked about, I mean, because that’s obviously one of the scalers that sort of helps.

      Murray: Yeah, yeah. So, basically, what’s driving the whole machine learning revolution, if we can call it that, is — I mean, there are three things. And one is Moore’s law, so the availability of a huge amount of computation. And, in particular, the development of GPUs, or the application of GPUs to this whole space has been terrifically important, so that’s one. Two is big data, or just the availability of very, very large quantities of data, because we have found that algorithms that didn’t really work terribly well on what seemed like a lot of data — you know, 10,000 examples — actually work much better if you have 10 million examples. They work extremely well. So, the unreasonable effectiveness of data, as some Google researchers call it, so that’s two. And then the third one is some improvements in the algorithm. So, there have been quite a number of little tweaks and improvements to ways of using backpropagation and the kind of neural network architectures themselves.

      Azeem: So, I add three more to that list. One is, in practical software architectures, we’re starting to see the rise of microservices. What’s nice about microservices — it’s a very, very cleanly defined system. So, you don’t need generalized intelligence, you just need very specialized optimizations. And as our software moves from these hideous spaghettis to these API-driven microservice architectures, you can apply machine learning or AI-based optimizations to improve those single interfaces. So, lots of reasons…

      Sonal: Right. That’s actually closely tied to the containerization of code at the server level, and there are so many connected things with that.

      Tom: So, it’s much easier to insert a bit of intelligence into a process.

      Azeem: And then the other two are — so there’s this phrase, which I’m sure Andreessen Horowitz is familiar with — which is, “software is eating the world.” And as software eats the world, there are many more places where AI can actually be relevant and useful. So, you can start to use AI in a food delivery service, because it’s now a software coordination platform, not chefs in a kitchen, and, therefore, more places for it to play. And this is a commercial argument, and so Murray’s explained some of the technical reasons.

      The third commercial argument is accelerating returns. So, as soon as you start within a particular industry category to use AI and get benefit from it, the increased profits you get, you reinvest into more AI, which means your competitors have to follow suit. So, you can’t now build an Xbox video game without tons of AI, and you can’t build a user interface without using natural language processing and natural language understanding. So, that forces the allocation of capital into these sectors, because that’s the only way that you can compete.

      Business vs. academia

      Sonal: So, given those six drivers, not three, who are the entities that are gonna win in this game? Like, is it startups, is it the big companies, is it government, universities?

      Murray: Well, if you were to ask me to place a bet at the moment, I would place it on the big corporations like Google and Facebook.

      Tom: Basically, they have access to the data, and everything else you can buy, but that you can’t, right?

      Azeem: Yeah.

      Murray: Right. And also, they have the resources to buy whoever they want.

      Tom: Right.

      Murray: An interesting phenomenon we’re seeing in academia these days is that it used to be the case that the people who, you know, were very interested in ideas and intellectual things — they wouldn’t necessarily be tempted away to the financial sector. But we’d still retain a good chunk of them in universities to do Ph.Ds. But now, companies like Google and Facebook can hoover up quite a few of those people as well, because they can offer intellectual satisfaction as well as a decent salary.

      Tom: But, also, they are getting the — you know, the Silicon Valley is the new Wall Street argument. They are getting the people who used to go into financial services, which is a good thing. I remember the head of a Chinese sovereign wealth fund saying a few years ago, you know, “You Westerners are crazy. You educate your people in these fantastic universities, and then you take the best people and you send them into investment banks where they invent things that blow up your economy. I think you have to do something useful.”

      Sonal: Right. We used to say that…

      Tom: And the whole of, you know, the Chinese politburo is they’re all engineers, and, you know, they value sort of engineering culture and engineering skills, and they can’t believe that we’ve, sort of, wasted it this way. So, I think it’s fantastic that, you know, now there’s less money to be made at Wall Street than maybe there is in Silicon Valley, and people like going West. I think that’s only got to be a good thing.

      Azeem: Coming back to who the winners might be, I mean, I think there is a strong argument to say that having the data makes a lot of the difference.

      Tom: Yeah, no, I think that’s the crucial distinction.

      Azeem: I think you’d be hard-pushed to say — look at voice interfaces, you know, between Apple, Microsoft, Google, Baidu, and Nuance. That’s quite a crowded field already, so it does feel like there are a lot of AI startups who are going to run up against this problem of both data and distribution. But, that said, there are particular niche applications where you can imagine a startup being able to compete, because it’s just not of interest to a large company now, and they may then be able to take a path to becoming, you know, independent.

      Tom: Look at, say, Boston Dynamics. Because one of the ways you train machines to walk like animals is not to use a massive internet data set of how cats walk. So, in that case, not having access to that data is not an impediment, and you can develop amazing things, and they have done. They’ve been acquired by Google.

      Murray: Actually, DeepMind are another example of the same thing. Because if you want to apply reinforcement learning to games — and that’s enabled them to make some quite fundamental sort of progress — you don’t need vast amounts of data, right? You just need to play the game loads, and loads, and loads…

      Azeem: We’re just reinforcing your thesis there, Murray, which is that Google’s gonna buy all of these companies.

      Murray: Well, yeah. Well, I ought to put in a little pitch for academia, yeah. Because the one thing that you do retain by staying in academia is a great deal of freedom, and the idea to disseminate your ideas to whoever you want — so you’re not in any kind of silo. And some of these companies are very generous in making stuff available.

      Azeem: Right, with TensorFlow, yeah.

      Murray: TensorFlow is a great example of that that we’ve just seen Google release. But, nevertheless, you know, all of these companies are ultimately driven by a profit motive, and they are gonna hold things back.

      Tom: We’ve just seen, for example, Uber has snaffled the entire robotics department for Carnegie Mellon. Presumably, the motivation of the people there is that, you know, finally, the work that they’ve been doing on self-driving vehicles and so on…

      Sonal: Right, you know, should get out into the world.

      Tom: And you can actually make a difference. And, yeah, I’m sure they get much better pay, but, I mean, the main thing is that rather than doing all of this in a theoretical way, here is a company that’s prepared to fund you to do what you want to do.

      Sonal: You can finally have impact.

      Tom: In the real world, in the next decade — and that must be amazingly attractive.

      Murray: It is incredibly attractive, and, of course, many, many people, you know, will go into industry in that way. But there’s also something attractive for a certain kind of mind in staying in academia, where also you can explore maybe some larger and deeper issues that you — I mean, for example, like, you know, Google aren’t gonna hire me to think about consciousness.

      Sonal: Or they might. You never know. I mean…

      Azeem: There’s also this question about the kind of questions that you will look at as an academic. So, the trolley problem being a good one. There are all sorts of ethical questions that don’t necessarily naturally play a part in your thinking when you think about your Wall Street <inaudible>.

      Sonal: That’s right. And corporate entities aren’t set up to think about that. Like, Patrick Lin studies the ethics of robotics and AI, and that entire work is funded by government contracts and distributed through universities. So, okay, so the elephant in the room — AI and jobs, what are our thoughts on that?

      Azeem: Well, I think look at where we are today, which is that we’re quite far away from a generalized intelligence. And, you know, McKinsey just looked at this question about the automation of the workforce, and they did something very interesting. They looked at every worker’s day, and they broke it down into the dozens of tasks they did and figured out which ones could be automated. And their conclusion was, we’ll be able to automate quite a bit, but by no means the entirety of any given worker’s job — which means the worker will have more time for those other bits, which were always the social, emotional, empathetic, and judgment-driven aspects of their job. Whether you’re a delivery person…

      Sonal: Right, the creative…

      Murray: Or creative, yeah.

      Tom: Yeah, and I’ve read that and I thought, “Hang on a minute though,” because what they’re looking at is, they’re looking at the jobs of basically well-paid information workers and saying, “Well, you can’t automate their jobs away.” But the bits you can automate are the bits that are currently — many of them are bits that are currently done for them by other people. So, the typing pool, you know, we got rid of the typing pool because we all type for ourselves.

      Sonal: Factory workers.

      Tom: Exactly. So, you know, this means that the support workers for those people are potentially put out of business by AI.

      Azeem: Or they’ve moved up.

      Tom: Yeah, or they have to find something else to do. But I think just because the architects are safe doesn’t mean that the people who work for the architects are.

      Azeem: If you walk down a British high street, the main street today, one of the things you’ll notice is a plethora of massage parlors, nail salons, and barbershops.

      Tom: Service businesses.

      Azeem: Because these are the things that you can’t do through Amazon. Everything else you can do through Amazon or Expedia.

      Tom: Interior design, yoga, Zumba, whatever. That’s the future of employment.

      Murray: And coffee shops.

      How far will AI go?

      Sonal: Okay. So, we’ve talked a lot about some of the abstract notions of this, and, you know, this is not a concrete answer, because we’re talking about a fiction film, but how possible in reality is the “Ex Machina” scenario? And a warning to all our listeners that spoiler alerts are about to follow, so if you’re really bitter about spoiler alerts, you should probably sign off now. The reality that the character — the main embodied AI, Ava — could essentially fight back to her enslavement. To me, the most fascinating part of the story — and we have no time to talk about it right now, but I do wanna explore this at some point in the future — is sort of the gendering of the AI, which I think is incredibly fascinating. How real is that scenario?

      Murray: Yeah. So, the whole film is predicated on the idea — well, it seems to be predicated on the idea that Ava is not only a human-level AI, but is a very human-like AI.

      Sonal: So, the humanoid aspect?

      Murray: Human-like, of course — she looks like a human, but, I mean, human-like in her mind.

      Tom: And her objectives.

      Murray: And her objectives and her motives.

      Sonal: Her needs, her emotions.

      Murray: You know, so if you were a person in those circumstances, you would want to get out, right? And, in fact, very often, science fiction films that portray AI — that’s a fundamental premise that they use for how they work — is that they assume that we’re going to assume that the AI is very much like us, and has the same kinds of motives and drives for good or for ill. They can be good motives or bad motives. They could be evil, or they could be good, you know? But it’s not necessarily the case that AI will be like that. It all depends how we build it. And if you’re just gonna build something that is very, very good at making decisions, and solving problems, and optimizing…

      Tom: It may just sit down and say, “I just wanna sit here and do math.” We really have no idea what their motivations will be.

      Azeem: Yeah, I mean, if the AI had been modeled on a 45-year-old dad, it would’ve been perfectly happy being locked up in its shed at the bottom of the garden with an Xbox.

      Sonal: And some of their magazines, right.

      Murray: Well, but then just moving on a little bit from that, though, it is worth pointing out some of the arguments that people like Nick Bostrom and so on have advanced — that you shouldn’t anthropomorphize these creations. You shouldn’t think of them as too human-like.

      Sonal: In the film — I saw it three times as I mentioned — on the third watching, I noticed that there is a scene where Nathan has, like, a photo of himself on his computer where he programs. Like, he’s on his computer all day, like, hacking the code — which I think is so fascinating because there’s almost this narcissistic notion, which kind of ties to your notion of the anthropomorphization of the AI.

      Tom: You use the term anthropomorphism because it is — I’ve noticed you use the word creatures to refer to AI, and I think that’s really telling, because they are going to be more like aliens, or more like animals, than they are like humans. I mean, the chances of them being just like humans are very small.

      Murray: We might try and, you know, architect their minds so that they are very human-like. But can I just come back to the Nick Bostrom kind of argument? Because he points out that although we shouldn’t anthropomorphize the AI, nevertheless, if we imagine this very, very powerful machine, capable of solving problems and answering questions, that there are what people who think about this refer to as convergent instrumental goals.

      Sonal: You’ll have to break that down for us really quickly, yeah.

      Murray: So, anything that’s really, really smart is gonna have a number of goals that anything is gonna share, and they are gonna be things like self-preservation and gathering resources. If it’s sufficiently powerful, then any goal that you can think of, if it’s really, really good at solving that goal, then it’s gonna want to preserve itself, first of all. Because how can it, you know, maximize the number paper clips in the world — to use Nick Bostrom’s argument — if it doesn’t preserve itself or if it doesn’t gather as many resources as it can? So, that’s their argument for why we have to be cautious about building something that is a very, very powerful AI, a very powerful optimizer. That’s the basis of the…

      Sonal: Because it will always be optimizing for that.

      Murray: So, I think the very important thing here is that the media tends to get the wrong end of the stick here, and think of this as some kind of evil Terminator-like thing. And so, we might think that those arguments are flawed — the arguments by Bostrom et al. Maybe we do, maybe we don’t, but I think there’s a very, very serious case to answer there, and in order to answer it, you have to read their arguments. You can’t just, kind of, assume what you think their arguments are.

      Sonal: Right, the derivative. That’s the problem with a lot of technology discussion in general is to always revisit these in a very derivative way, versus viewing the original. But putting that exhortation aside, how do people make sense of this? Like, how do they make sense of what is possible?

      Murray: So, how do we think about the future, really, when it comes to artificial intelligence? And I think the only way to do it is actually to, kind of, set out a whole tree of possibilities that we can imagine and try to, you know, not sort of fixate on one particular way that things might go — because we just don’t know where we’re gonna down that tree at the moment. So, there’s a whole tree of possibilities. Is AI gonna be human-like or not? Is it gonna be embodied or not? Is it gonna be a whole collection of these kinds of things? Is it gonna be a collective? Is it gonna be conscious or not? Is it gonna be self-improving in this exponential way or not? You know, I don’t think we really know, but we can lay out that huge range of possibilities, and we can, you know, try to analyze each possibility and think, you know, what would steer us down in that direction and what would the implications be.

      Sonal: That’s a great way to approach it. Well, that’s another episode of the “a16z Podcast.” Thank you so much for joining, everyone.

      Azeem: Thank you.

      Murray: Thank you.

      • Azeem Azhar

      • Murray Shanahan

      • Tom Standage

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Beyond Lean Startups

      Eric Ries, Sonal Chokshi, and Michael Copeland

      What began as a scientific approach to creating and managing startups has now become a worldwide movement for companies of all sizes — and for creating (or rather rediscovering) entrepreneurs in all places. Not just inside startups, not just for software, and not just inside Silicon Valley. It’s about unlocking human creativity everywhere. Perhaps even reinventing the firm.

      As utopian as that sounds, Eric Ries — who pioneered the lean startup movement and wrote the definitive book on it — argues the case in this episode of the a16z Podcast. But has it become too much of a religion? One where people apply the letter of, but not the spirit, behind lean startup principles?

      Ries, who recently crowdsourced a leader’s guide for practitioners to test and evolve the very concepts he first published 5 years ago, shares lessons learned — as well as the true meaning of overused terms like ‘MVP’ and ‘pivot’. Ultimately, lean startups are about how to make decisions and build new products under conditions of high uncertainty. Without having to chisel the principles into stone tablets.

      Show Notes

      • Where the idea of “lean startup” came from and how it’s resonating with larger businesses [0:00]
      • How innovation is often stifled in large organizations [10:22], and why it’s difficult to be entrepreneurial [20:23]
      • The impact of software [27:31] and keys to unlocking people’s creativity [37:16]
      • How to implement lean startup, and a discussion of the most popular terms [41:52]

      Transcript

      The origin of “lean startup”

      Sonal: Hi, everyone, welcome to the “a16z Podcast.” I’m Sonal, and Michael and I are here together interviewing Eric Ries, who wrote the book on “The Lean Startup.” And it’s been, actually, now over five — is that right, Eric? About over five years since that book came out?

      Michael: Yeah, it came out in 2011. So, next fall will be the five-year anniversary of its publication. And of course, the publication was the, like, the end of a process of blogging and writing and, kind of, building that community.

      Sonal: I was about to say because I feel like your book came out well after lean startup was already on its way. Well, the other interesting thing that I think we’d like to talk to you about is how things have changed since then and now, but more importantly — we’d love to also — because one of the things that’s happened is that lean startup is now outside of Silicon Valley. And it’s gone also outside of startups, interestingly. So, we want to hear a little bit about your insights from that. And then kind of talk about not just the lean startup, but what it means for the future of the firm.

      Michael: What is a lean startup? And if it’s not a startup, how does that apply here? 

      Eric: So, the basic idea is to take a more scientific, more iterative, and more customer-centric approach to product development and customer building. It’s called lean startup, because we take ideas from lean manufacturing around cycle time and batch size and iteration, but we apply them not to the factory floor, but to the process of innovation itself — where the signal that pulls work from us is not the order from a customer, since in most startups, we don’t have a customer yet. We don’t know who the customer is going to be. The pull signal is our hypotheses, our beliefs, about what the customer will want in the future. Or another way of saying it is, we write these business plans that are full of assumptions and hypotheses and guesses about the future. And lean startup says, “Rather than take those things for granted and hope that they’re true, let’s test them scientifically — accepting the fact that every startup faces high uncertainty about the future.”

      Sonal: So, how do you respond to critics who say, like — I would say, actually, thinking about it more ethnographically, that you’re essentially only then building for people who know what they — like, it’s based on what the market is getting as feedback versus, like, true internal creativity that’s an internal compass.

      Eric: Yeah, it’s an interesting thing, because customers don’t know what they want. And everyone knows that to be true, or talks about it. And if you imagine I’m a scientist, I came in, “I’m gonna do chemistry,” and then I’m like, “Oh, shoot, the electrons don’t know what they want. And I can’t do a focus group with protons. Like, forget it, I guess I can’t do chemistry.” Like, that person is bonkers. <inaudible> mental problem. For some reason, we talk about customers, we get confused about this. Taking a scientific approach means having a very strong belief — some would say a visionary belief about what’s going to work — and finding out if that vision is right. And what we discover with — if you look at the stories of entrepreneurs, not the kind of movie version and the Hollywood version that you read in the magazines, but we get the real stories, you know, with actual entrepreneurs — what you discover is that even if the vision is right on, and they see the future, the specifics of the strategy often have fatal flaws.

      And until you systematically figure out which elements of the strategy makes sense, right? Do we have the right business model and the right target customer and the right, you know, approach to disruptive innovation? Are we on the right technology platform? Have we gotten our timing right? You have to work to get all those details right. And the best entrepreneurs I know are extremely rigorous in their thinking. Because — it’s precisely because they care so much about the vision, they feel like they have to get the details right. And so all we’re saying is that we should approach that with the most, you know, scientific kind of rigor that we can.

      Sonal: Right. Essentially, a business plan is a hypothesis.

      Eric: That’s right.

      Sonal: You can, sort of, test out not just whether it works or not really, but actually the details of how to execute on that because the vision is a large thing.

      Michael: So, why is it then — we talked about this was the five-year anniversary, how is it that this has resonated so well outside of the Silicon Valley startup world? You know, what are the things that people were, kind of, grasping to, whether it’s aspirational or where you’re like, “I can roll up my sleeves and do this?”

      Eric: Yeah, you know, it’s a question I think a lot about, and because no one is more surprised than I am about how this has grown beyond just a book and beyond just a set of like idiosyncratic ideas into a movement.

      Sonal: It’s like a movement for a lot of people. Right.

      Eric: You know, I feel like I have to disclaim periodically that lean startup is not a religion. Like, if you have to say that then something’s a little suspicious here. You know, I think that’s fair game. I think that, first of all, you gotta look at the timing of it. I started writing about this idea in 2008, 2009, right in the height of the financial crisis, when people had kind of lost confidence in, kind of, traditional ideas about how a company should be built. So, there was a — kind of a hunger for something new. I think having it called “lean startup” at a time when people were freaking out about money was probably good timing, good branding. And I think that as entrepreneurship has become more democratized…

      Sonal: What do you mean by entrepreneurs have become democratized?

      Eric: So, you know, I think there was a time when if you wanted to become an entrepreneur, you had to have the right connections, you had to have access to a lot of money and capital, you had to, kind of, look the part. And, you know, if you look at the, like, famous entrepreneurs in history, a lot of them came from very specific kinds of backgrounds. And it was a rare thing. It was considered a little bit crazy. You know, as the semiconductor revolution has, kind of, systematically, you know, eradicated barriers to entry for everything. I think about it — you go back to the old Karl Marx idea of who owns the means of production.

      We are now living through a time where you can rent the means of production. Which means that if you want to test an idea, if you want to start something in your dorm room at Harvard, you know, just because you started small doesn’t actually mean it can’t scale to something, you know, massively large. And so, you have that phenomenon. Plus, the fact of the internet itself has made the idea of entrepreneurship as a possible career path accessible to people who, you know, used to have to, like, know somebody in Silicon Valley to, kind of, like, understand what was happening here. Now, Silicon Valley is like a reality TV show that everyone in the world watches.

      Michael: I want to go back to Karl Marx, just for a minute. You say who owns the means of production? Sounds like Amazon does. But maybe owning the means of production isn’t worth as much as — well, it’s worth a lot to Amazon, but…

      Eric: Yeah. Well, I think the kind of older industrial thinking was that if you own something, then the right thing to do was to control it in order to maximize the return you would get from it. And I think what we were having a recognition — probably partly through the just the realities of how technology works. But I think also partly due to, you know, like better management thinking, frankly — is that if you make those tools available to other people so that you don’t have complete control —  if you turn it into a platform, if you give people open access — the sum total of the creative power that you unleash when you do that means that you have a smaller piece of a much larger pie.

      Sonal: So, speaking of that shift in management thinking, let’s talk about the surprise that you have and that we’re surprised by — that big companies are trying to be lean startup-like. I mean, because I think one of the things that’s interesting about the management point is that we’re living through an interesting time right now, where a lot of the old school theories about management — like, they may apply in certain ways but in other ways, there’s a real gap — a real need for, like, how do you really — the world has changed, and it’s very trite to say that, but it’s true. So, talk to us more about that.

      Eric: I’ll tell you a funny story. So, when I was doing the research for “The Lean Startup,” since we’re talking about the five-year anniversary — doing the research for the book, I read everything I could get my hands on about management. Business books. I felt like I needed to really understand what came before. And I remember reading about the development of our modern accounting system, which I didn’t really know a lot — I’d never thought about, what — going back to the 1920s and Alfred Sloan and that whole movement. And I realized at a certain point that we had developed this accounting system not to keep track of the money, which is kind of how we think about it now — it was originally developed as a system of accountability, so you could figure out which managers are really doing a good job. Because if some manager makes more money this year than last year, they say, “I should be promoted.” But you’re like, “Hold on, if you had a forecast of how much money you were supposed to make this year and you fell short of the forecast, you don’t get promoted for that.”

      We take that so for granted in our world today. But when I first read that, I almost fell out of my chair, because I’m, “Wait a minute, I’m sorry. You’re telling me there’s a part of the world — or a part of history or anywhere — where people make forecasts for things and then they come true? I never heard of that.” You know, as an entrepreneur, I had no idea why I was always asked to make forecasts. I thought the spreadsheet I put in my business plan when I raised VC was like a kabuki ritual I did to show how much pain I could endure — to show that I was tough. I didn’t think anyone would read the content of those spreadsheets and take them seriously. We just made those numbers up. We had no idea. But of course, if you, like, think about that — like, that is so specific to my experience as an entrepreneur. And I was like, “Okay, well, how is it that I’ve never seen an accurate forecast, but in the real world, in mainstream business, forecasts rule the world? Why?”

      And so I, you know, studied that and learned about it. My realization was, like — a forecast is accurate only to the extent that it is an extrapolation from a long and stable operating history. So, any time you don’t have that — either because you’re doing something brand new, or because your long and stable operating history has just gone unstable on you, and, “Oh, my God” — then you’re in a situation of high uncertainty. And since my definition of entrepreneurship is trying to create something new under conditions of extreme uncertainty, then you are an entrepreneur, no matter what it says on your business card. Now, when I first started saying that out loud, you know, in talks and conferences, you know, a couple years ago, this weird thing happened to me where people would come up to me after the talk. And they would say, “Oh, hi, I’m a general manager at such-and-such large business, and I accept your challenge.” 

      First time it happened. I was like, “What are you talking about?” <What challenge?> They’re like, “What you said because of the definition — that this can apply to companies of all sizes, all sectors, all industries.” I was like, “Yeah, I just said that.” So, they’re like, “I would like to go, you know, prove that that can work.” And [the] first time it happened to me, I was like, “Good luck. What does it have to do with me? Like, okay, that sounds great. This is just a deduction from a theory. That’s not my responsibility.” But then, luckily for me, really, some very visionary, you know, big company folks, kind of, dragged me kicking and screaming to the realization that there are real honest to God entrepreneurs — just as visionary, just as exciting to work with — inside some of these larger organizations as you walk down the street here in Silicon Valley.

      Michael: And it is that lean startup, kind of, army being activated because of the conditions now facing these large companies, I mean, that you describe?

      Eric: I think that’s right. I mean, look, big companies have always faced the forces of disruption. That’s an old phenomenon. But the rate of change and the, kind of, existential crisis that some of these companies find themselves in, I think, is more severe than ever. And there’s more of a recognition that they need entrepreneurial activity in order to survive. So, I think that has created a fertile ground for these ideas to come in, because so many companies want to act more entrepreneurially. You know, the idea that you have to act like a startup, or have internal startups, is, like, almost to the point of cliché here. And yet most of the companies that I meet that have that as a plan, there’s no plan. They don’t actually know how to make it happen.

      Innovation and big business

      Sonal: Right, exactly. But I do want to say that they do have long standing processes. I mean, R&D — deciding about where to invest your resources — that’s all about decision making under uncertainty. And there are actually entire schools of portfolio management for managing R&D around that, which — some of it is very not dissimilar to managing a portfolio as a VC, with a lot of startups. And so, I think I want to pause for a minute what you said about things happening faster and just make sure we reflect on what that really means. Because, that’s a phrase people use, like, “Rapid changes now. Things are dynamically evolving.”

      What we’re really saying is that these big companies, which before could have quickly acquired startups to help them do some of these things — now those companies get too big. It reaches a point where their market cap is too big — we’ve observed this — for them to even consider affording the ability to then take on that company. So, it’s a really big deal that some of these companies can’t then innovate themselves. So, it’s a big problem you’re talking about is how to get at that. So, anyway — so, given this condition like what have you seen about how people are becoming — I mean, isn’t there a word called intrapreneurs? That’s been around forever? Like, what does that mean?

      Eric: Yeah, you know, I don’t actually like that word that much…

      Sonal: I don’t either.

      Eric: …because I feel like, you know, an entrepreneur is an entrepreneur. It doesn’t matter if they live in a garage, or they wear a suit, or they have health benefits, or if they eat ramen noodles. Like, the surface details don’t matter. What matters is the fundamental, you know, reality of their job, which is — they’re trying to create something fundamentally new. And, you know, a lot of big companies actually have outstanding research labs, where they’re doing breakthrough science, and they manage the scientific uncertainty really well. And yet, as soon as they take those discoveries out of the lab, it’s like, “Okay, we’re done with the science, now the astrology.” And it’s like, they take these world class scientists and, like, “Forget everything you know about science, now we’re going to build a business plan. Tell me what’s going to happen in the future, and then make it happen through the power of your mind.”

      Sonal: And that’s a great analogy from science to astrology.

      Eric: And the scientists are like, “What are you talking about? That doesn’t make sense to me.” I’ve now worked with a lot of big companies, and a lot of companies with high science research labs. I meet these scientists, and I say, “So, tell me about some of the great breakthroughs you’ve had in the lab.” And they’re very excited to tell me about it. I say, “Great. Tell me which of those have been commercialized and are in products today, and which ones are sitting on a shelf.” And now you may as well start the violin music, because it is really depressing. Life-saving treatments, unbelievable breakthroughs, sitting there. It’s like, these companies can spend, through their technology readiness level, you know, analysis and — like they can do the smart research to spend $5, $10, $50 million to have a breakthrough. And then they often are not able to spend the, like, $2 million extra dollars that would be necessary to commercialize it, because they’re organized around functional silos. And there’s nobody whose job it is to actually take it out of the lab and make sure that the businesses that operate — that are mostly tied to quarterly short-term incentives, you know — have the ability and the incentives and the time and the space to figure out how to commercialize.

      Michael: How do you mean that it’s nobody’s job to do that? I mean, $2 million —  that should get done, and then we’re off and running, right?

      Sonal: It’s, like, everybody’s job.

      Eric: Yeah, it’s everybody’s job, which means that it’s nobody’s job. My observation is that in most companies, there’s a missing function for entrepreneurship. So, there’s just nobody in charge of making sure that new ideas are taken from concept to execution. There’s not a disciplined, systematic way of testing new ideas. So, I used to think — I’m a Silicon Valley person, quite arrogant about the world. Our way is the best. You know, I used to think if I sat a big company person down, I said, “Hey, do you have any ideas for how your company could be better?” That they wouldn’t have any good ideas, because people are dumb if they work in big companies. That’s what I used to believe. And, you know, what I’ve learned the hard way is that that’s actually a dangerous question to ask, because you gotta have four or five hours to spare to get the answer, because you can’t shut people up. They got tons of amazing ideas of things the company could do better.

      Sonal: Totally. The biggest thing is that disruptors always know what’s coming. It’s not like they don’t know what’s coming.

      Eric: Yes, definitely. The information and ideas are in the company already, and the talent is in there, too. If you want to shut them up real quick, just say, “Okay, tell me the process to test out those ideas to see if they’re any good.” And they’re like, “I guess I got to ask my manager to ask their manager to go across the silo to the other manager.” So, like, visualize — you put the idea in a pneumatic tube, it gets sucked up the org chart somewhere, sent somewhere else, sent down and they’re just like, “Forget it.”

      Michael: Because nobody pays attention.

      Eric: No one pays attention. It’s, like, just discussing the process is, like, so painful. They’re like, “Forget it. I’m just gonna go back to doing my job.”

      Sonal: But isn’t that…

      Michael: So, here’s the thing, it sounds to me like these large companies are coming to you, in some sense, for youth. Like, you see in the movies where, you know, the witch is sucking the youth out of a child. But isn’t it the natural course that these companies — they get big, they get old, and they get plowed under?

      Eric: I mean, that’s a very common belief in Silicon Valley.

      Michael: But are you gonna change that?

      Eric: I basically used to think that, too, but I don’t believe it anymore. I think that — you know, I come in as a consultant, or — so I come in as an outsider. And one of my strengths is, I don’t run a consulting company, that I don’t have 50 associates I’m trying to feed. I come in and I can tell companies the truth. What I tell them is, “Listen, as a consumer of products in your category, I don’t care if you live or die. I know that 5 or 10 years from now, the person who provides me this service — it’s going to be technology-enabled, it’s going to be developed according to these principles, it’s going to be rapidly evolved to suit my needs. So, as a consumer, I’m fine. Either because you will have adapted to that new reality, or some startup my friends down on Sand Hill Road are funding right this second will disrupt you and displace you, and I don’t personally care. So, whether you live or die.”

      Now as a consultant, that’s not generally considered a nice thing to say, but it helps, because people who don’t want to hear that kick me out of their office and we save ourselves a lot of time and heartache. The ones that have been willing to say, “Okay, what would it take to do the transformation,” and it’s hard — I think I’ve seen really dramatic results. So, I have become a believer that even — that the bureaucracy, and slowness, and, kind of, ossification that we take for granted as a result of scale is not an inevitable development, but is a choice about the systems of management that we use.

      Sonal: So, can you tell us a little bit more then about what you’ve seen on a big company side? Because frankly, I think, yes, you’re right — there isn’t, like, a chief entrepreneurial officer that owns a function, or the process for that matter. But there are groups within a company that try to — like, they have weird titles often, which is probably also a sign of not a good thing — but that do own this in some way, shape, or form. I mean, how do you prevent the risk of that just being yet another idea that doesn’t go anywhere? Like, how does lean startup, kind of, help with that? Like, what have you seen on the front lines of that?

      Eric: Someone once came up to me after a talk and they said, “I have a question for you. There’s this guy in my company who has a C-level title.” I think his title was, like, chief innovation officer or something. He said, “That guy always comes to work in red pants. He has no responsibilities. He doesn’t do anything, as far as I can tell. He has no operational — he’s not responsible for any quarterly targets.” He’s like, “If I came into work dressed like that and talking like, I’d be fired in a heartbeat.” He’s like coming up to me like, “Can you explain to me what this person does?”

      Sonal: That’s so funny. It’s like therapy.

      Eric: Yeah, the guy in the red pants. “You mean that guy in the red pants?” You know, I was like, “Like, don’t blame me.” Like, paying lip service to innovation is easy. Doing it is really hard. And the question I always have is, like, if I want to find an entrepreneur inside a large organization, I can usually go to the middle manager. I say, “Listen, I got this kind of wacky, crazy project. Do you know a lunatic who would be dumb enough to sign up for this suicide mission?” They’re like, “Well, let me show you my secret black book.” There are these certain people that are known in the organization. If I pull their personnel file from HR, they are full of black marks. “Does not play nice with others.” My favorite is, like, “refuses to obey the standards.” They defy the standardization of work and that drives people crazy.

      In a lot of companies, they get fired and bought back more than once, sometimes, for their startup. And it’s just, it’s crazy. Like, these people exist. Then it’s like, “What’s their job title?” In some companies, they’re a product manager, they’re an engineer, they’re a marketer, whatever they are, they — like, how do they get promoted? If they’re good at what they do, if they were really good entrepreneurs, how do they get promoted? Where do they live in the org chart? Who do they look up to? And what I’ve been working on lately — I’ve been thinking about — is, like, a grand unified theory of entrepreneurship, which is this. In most companies, including, by the way, startups that have gone through hyper growth…

      Sonal: Like Google. I mean, how many people leave Google to start startups now?

      Eric: I mean, it’s unbelievable. You have basically four completely different jobs that, to me, are the same. <Okay.> You have somebody who’s, like, a product manager, tasked with leading on brand new product development. So, you say, “We’re going to enter a brand new market with something that’s radically different. We’re going to try to be the disrupter for once.” You’re that person. You have someone in charge of a new internal system. And think about how many new IT systems you spend years and millions of dollars on, and they’re dead on arrival. It’s like, it’s the same old waterfall development. It was the old school Silicon Valley way — too much money, too little customer feedback, too long development. That’s true for new HR policies, new finance policies. I mean, you name it. That’s actually an entrepreneurial challenge too.

      Then you have somebody in the business development, you know, office who’s supposed to be evaluating outside startups for purchase, and they make these catastrophic errors. You know, they’ll buy a startup for $900 million, and 3 years later, it’s worth $15 million. You know, like that — in most parts of the corporation making catastrophic errors like that would get you fired, but in Biz Dev, it’s like, we don’t know how to ask the right questions to figure out who’s doing a good job and who’s not. We wind up flooding the entrepreneur ecosystem with dumb money. And then you have people who are responsible for partnering with startups. Most big companies are terrible partners. They don’t understand how to pilot things. They don’t understand how to work with startups in a way that don’t kill them. They spent way too much time on contract negotiations, and they just — they’re unreliable partners.

      Sonal: Oh, and then you have the “not invented here” syndrome, which pretty much kills anything…

      Eric: Right. Totally terrible.

      Sonal: …if you have an internal group of any sort.

      Eric: So, what all those jobs have in common, to me, is this entrepreneurial reality — that they deal with situations of high uncertainty. And therefore, we need discipline as a company to be able to look at what are the right metrics to hold those people accountable? How do you identify who’s actually good at that job and who’s not? How do you share best practices across these similar things? So, you start to add up these tasks — a career path, a sense of professional pride and accomplishment, standardization, you know, having the right metrics — you’re like, “Gosh, that sounds a lot like a corporate function, right? That’s what — we do that in marketing. We do that engineering. We do it in R&D.” And people are like, “Well, entrepreneurship is too creative to be managed.” But it’s like, if we can manage R&D, like, we can manage Muppet Labs. You know, people working on [a] Nobel Prize, they can be managed.

      Sonal: That’s right.

      Eric: So, I just — I don’t buy it. I think that we have just made a mistake about how the companies were organized, so that we can pay lip service to innovation and claim we want to have continuous innovation. But I ask these CEOs that I meet with all the time, “Who’s in charge of making sure that that happens?” And they don’t know. There’s nobody in the organization they can point to for accountability on that score.

      Lean principles for large organizations

      Michael: So, what are these organizations that you talked to who are, you know, stuck in the present and perhaps in the past? You know, when you think about the organization of the future, what does that start to look like? You know, when you look at the org chart, or when you look at it, sort of, structurally otherwise?

      Sonal: And just to pause there for a second, I think we’re not just asking about, you know, “What is lean startup applied to a big company?” It’s really about reinventing the nature of the firm.

      Eric: Yeah. So, people talk about lean startup for startups and then lean startup for the enterprise, which I think is really silly. I understand why people do it, but it doesn’t make sense to me. And the reason is, only the bad startups are small companies, right? The people are unintentionally small companies, but that’s not what they’re trying to do. And I meet all kinds of entrepreneurs who became an entrepreneur because they hate working for big companies, and they find them bureaucratic and sclerotic. And I always ask them the same question, I say, “Listen, if you hate big companies so much, why are you trying to create a new one?”

      And what happens is, five years later, they have all the success, they achieve product market fit — you know, in the blog posts, in the books — you know, in everywhere except for Ben’s book. It sounds like when you get product market fit, all your problems are solved. But you know, like, the reality is, everything gets way harder. And the curse of it is, you have these founders who I meet with all the time now, who have 100 to 500, 1,000-, 5,000-person organization — and they’re like — you know, you’ve got to get them privately off the record. I’ll be like, “I’m not sure I would even want to work here.” I mean, I got a good gig because I’m the founder CEO, and that’s pretty fun. But, like, if I wasn’t, would I actually want to, like, be a regular employee here? And if I was trying to do something entrepreneurial here, would people understand how to do it? And they don’t… 

      Sonal: Even the founder CEO is, like, desperate to hold on to that feeling of how it was in the first year, first five years.

      Eric: Right. Yeah, they can feel the loss of it, and they feel the frustration because people come to them with plans. I mean, I was just talking to a very famous recent mega success story. And they’re telling me this unbelievable story, where the founder was being pitched on a new app, you know — the new big line of business for them. The founder was like, “Okay, that seems like a pretty reasonable experiment. It should take about two…” And he’s like, “I could probably code that in a week or two weeks.” He’s like, “I can do it in a week. So <inaudible>.” And the team was pitching him on, like, a 12-month, multi-million dollar, like, mega plan that’s, like, overengineered to the max. And he’s just like, “What have I done? How can I possibly have a company where people think that’s a good idea?”

      And the challenge for me, you know, talking to them is to be, like, “Look, I hate to be the bearer of bad news, but you need to look in the mirror. And now you’re looking at the problem, because you have to make a fundamental choice as any kind of leader. Are you trying to preserve that entrepreneurial feeling for yourself, or are you taking the steps necessary to push that entrepreneurial opportunity down into the ranks of your company?” And, in fact, most of the CEOs who are good at this realize that they are so used to being on one side of the accountability table — they’re the entrepreneur pitching on their board, and their VCs always asking them about progress and having that negotiation — what they don’t realize is that now the roles are reversed.

      For the people inside their company, they’re the VC. They’re the source of funding and political capital that everybody needs to sustain what they do. So, when they’re being pitched crap, it’s the same as — I know a lot of board members in a lot of companies that are like, “Why are these companies always give me these stupid reports and these dumb updates?” Like, because that’s what they think you want to hear. So, if you want them to do something different, you have to be the one to say, “Here’s how I intend to hold you accountable.” And then, when we have that real conversation, a lot of these entrepreneurs, they themselves have amazing intuition and really good natural instincts for, like, what are the right metrics to look at? You know, they all, kind of, naturally gravitate to the minimum viable product. And the idea that, you know, a small number of extremely passionate customers is way better than a large number of people who kind of are indifferent about your product. But they don’t understand why the people that work for them don’t have those instincts.

      Michael: So, what’s the hard — I mean, it all sounds pretty hard, I’ll be honest. But, like, what stands in the way? Like, again, if I know what I need to do, if I’ve done it before as a startup, now I’ve — my startup has grown and successful — like, what stands in the way in the companies that you talk to from them actually realizing their entrepreneurial, kind of, style and flavor and goals?

      Eric: Yeah, the problem is strictly scale. So, the founder — they can, kind of, go on a side project and be like, “Forget it, I’m going to do it myself. And I’m gonna step out of the CEO chair and go show this project team what to do.” But they can only really do that one, like, at most one project at a time. But these companies are too big for that. They need to be doing — you know, if you want to have a new successful disruption every couple of years, you need to have hundreds of experiments going at any one time. So, then — it’s like, you need to have a way to train and reward the entrepreneurial people in your organization. 

      And they have natural instincts for that. But that’s really different from saying, “How do you teach that approach to other people.” And that really, I think, is why lean startup is taken off inside these larger organizations — which, by the way, is both legacy organizations that are, like, 100 years old and are now adopting it recently, as well as these companies that started as lean startups but then blossomed into this traditional company.

      Sonal: And, by the way, by organizations, that includes governments.

      Eric: Oh, yeah.

      Sonal: Because I’ve heard like the former CTOs of the United States talking about how they’re trying to adopt a lean startup-like methodology inside of government.

      Eric: And it’s amazing. I mean, I was actually just in D.C. the other day and meeting with teams. They showed me this almost unbelievable story about this team inside of the Immigration Service that processes applications still on paper. And the paper applications can’t be stored in a normal office building, because the backlog is so large that the physical weight of the paper requires a structurally reinforced room. And, therefore, at one of the processing centers — I think they said in Kentucky, the processing center is literally in a cave. Like, that’s not a metaphor. Now, they have historically had these big outside contractors come in and do these multibillion dollar IT initiatives. They were telling me about one they spent, I think, a billion dollars in seven years, and it couldn’t process even one form faster than paper.

      Michael: But the great thing is the solution was to move to a cave.

      Eric: Right, right. I always think about the human cost of these bad management systems. Think about the poor people actually trying to get this work done the best way they know how, and that’s the best they could come up with. They sent a lean startup team in — I think from the United States Digital Service at the White House. And, you know, they partnered — it wasn’t just IT people coming and telling everybody what to do. But they had real partnership, real user-centered design, real lean startup experimentation techniques. And they built a small team of technologists and people who are experts in the processing center. And they’re now processing something like 40% of the applications digitally. I think it took, like, six months.

      So, instead of spending a billion dollars in, kind of — I call it the healthcare.gov plan. Instead of executing the healthcare.gov plan, we did something a little bit better. And I love those stories, because when I meet with private sector folks — I meet a lot of CIOs now, and they’ll tell me about some new major initiatives they have going, and I’ll say, “Oh, that’s great. Sounds like the healthcare.gov plan. I’m sure you’d be fine.” And they’re like, “How dare you suggest such a thing? That’s government.” I’m like, “Listen, let me draw a little chart. Here are the things they did and healthcare.gov. Right, big upfront design, no customer, no iteration…”

      Sonal: RFP, multi-stakeholder consensus, blah, blah, blah.

      Eric: <crosstalk> And let me show you that chart for what they did and what you’re doing. So, what’s the difference?” And they get mad at me. But I said, “Look, the truth is, that system of managing work is not a good one for our time. Maybe it made sense in a different era, but it really doesn’t make sense now, and there is a better way.”

      Impact of software

      Sonal: What you’re really getting at — and I don’t think this is as evident to people who aren’t necessarily inside the software industry — is that, in a lot of ways, lean startup is almost synonymous with the world being eaten by software. Because it’s really about a mindset for how people move fast, have a certain methodology, the ability to democratize, as you talked about, the ability to be agile — whatever all those adjectives are, they actually have meaning. They’re buzzwords, but they have meaning. Tell me more then about how software reinvents the firm as a consequence of this.

      Eric: It’s really interesting, because — I’ll tell you two stories I think are actually — one, I was talking to a company that was really struggling with its agile transformation. And forget lean startup, they’re still trying to get their software that they write. They employed lots and lots of software developers. They’re trying to get everyone to go agile. And I was meeting with the folks at their — in the software part of their business. And they said to me, you know, “We’re really having a hard time getting the non-software functions in the company to do agile. We’re doing pretty good in the engineers, but like getting people in the hardware, manufacturing, supply chain, but also HR, finance,” they’re like, “Straight up leadership managers to understand agile,” they’re like, “Like, I don’t want to do some, like, software thing.” Lean startup has created a neutral terrain where different functions can come together to do this. And I think there’s just a natural resistance to doing a methodology from someone else’s function.

      So, like, if you ever tried to get software engineers to do design thinking, or try to get non non-manufacturing people to do lean manufacturing, you know, you try to get non-operations people to do DevOps, it’s like people are gonna be like, “That’s not my thing. That’s their thing.” And what are they trying to do? The lean startup is a neutral terrain. It is, you know, not associated with any one specific function and therefore — and it’s denominated in terms of business results only. So, people talk about, “Oh, we show people in finance our burndown chart, and they can see how fast we’re making this in.” People in finance don’t care, like, “Well, how much money am I gonna make?” And, you know, we have this thing in a startup called innovation accounting, which is a formal methodology for translating what we are learning about customers and our business plan into financial performance results that give us leading indicators and confidence about the future. It’s a very important part of the method.

      So, that’s one thing is — is that although this is software enabled, it is not a software-specific thing. But the point you made that is exactly right — and this is true for every kind of software or semiconductor-related change — it really is about mindset more than tools and materials. And I’ll tell you a funny story. I was working with a consumer electronics company, and they were building this new device. For their confidentiality, I won’t tell you what it was. But whenever I work with hardware — I’m a software guy by nature. I grew up in my parent’s basement programming computers. Like, that’s me. So, whenever I deal with hardware things — because I’ve worked with, you know, the GE’s of the world, the Toyota’s of the world on big physical — if it can explode, I tend to be very humble. You know, I was like, “The nice thing about software is it doesn’t tend to explode.” I always appreciated that. So, I was working with this team, and I said, “Gosh, it’s probably going to be very hard for us to build a product.” We had to build a minimum viable product. Instead of building 10,000 units or 100,000 units, what would it take to create 5 or 50 units, or even 1 unit?

      And I was like, “Gosh, that’s probably gonna be really challenging.” And of course, whenever you have engineers in the room, once you frame the problem correctly, they’re like, “Oh, that’s no problem. We’ve actually already done that. We were just playing around with some 3D printers and soft tooling. We have a prototype of it in our office right now.” Well, then the problem is, how do we find an MVP sales channel? You’re not going to get Walmart to carry some unknown prototype device. We’re probably gonna have to find a local store. So, I start to, like, “How can we find a place we can get…” They’re like, “Well, actually, we run a model store in our company, you know, where we can showcase new technologies for customers. So, customers are in there 24/7, you know, looking at new things.”

      And I said, “Oh, well, the problem must be, then, that that store is really far away.” I’m like, “How do we get access to the store?” And they’re like, “No, it’s in the same building where we work.” I was like, “Okay, is it like on a different security system? You don’t have the right badge to get down there? Like, is it a different team that operates?” They said, like, “No, we operate ourselves.” And I was like, “Okay. Do we need, like, a dolly or something to move the thing the hundred yards from your office to the model store?” And they’re like, “No, we could just pick — it’s heavy, but the four of us could easily carry it there.” And I was like, “Okay, timeout. You have everything you need. You have the <inaudible>.”

      Now, if you look at that story, why do they have a model store? Why were they able to produce this prototype? Like, there’s software lurking in the edges — “here there be dragons” in that story, in many places, but it’s not really about software. It’s about the fact that the company has the capability to work in this new way, but it had never occurred to them to do it. When I said, “You should take this thing out of your office, walk 100 meters to the store and offer it for pre-sale to customers,” They were like — they thought I was crazy. They looked at me, like, with wide eyes, like, “What are you suggesting? That seems nuts.” But then we really walked through the method and walked through the, you know, reasoning from first principles about why that would be a good idea, and it revolutionized their business. And they eventually got to a place where they can show — they can iterate on this so quickly now. They can build a new version of that device every week. Used to take them three to six years to build a new model, now they can do a new version every week. So, they’re constantly getting testing and iteration with customers. And, like, that’s not gonna scale up to a million units, but that could get us — you know, it’s like, well, if customers don’t ever want to buy the thing, we’ll never have to scale it to a million units. We just saved ourselves a lot of costs and time and energy.

      Sonal: So, like, what’s the high-level moral of that story?

      Eric: So, there’s a couple things. One is just people use it as an example, I think, to see how a minimum viable product thinking can work, right? So, reduce scope, reduce the number of customers affected, try to figure out — what is that experiment that can help us learn whether our strategy is actually right? And in this case, what they learned was that what they thought customers wanted was wildly different than what they actually wanted. But the other moral of the story is that this, before it is anything else, is a management issue. It is not a technology issue, it is not a process or tools issue. It is really about “how do we manage people?” Most companies have all the raw material they need to work in this more entrepreneurial way — including, by the way, the actual people that you would need to act in this creative way. I mean, I’m amazed at the caliber of people who work at these companies who are being told what to do.

      Sonal: You’re right. And in fact, I think people are born and really do have — I always think of this analogy of children always coloring when they’re little.

      Eric: That’s incredible.

      Sonal: Like, who beats the coloring out of them? I mean, now there’s actually this trend where there’s all these adult coloring books, which is something in and of itself. But the point I think is that that creativity never dies. It never dies.

      Eric: Yeah, that natural creativity. I’ll just tell you one more story. It’s such a cliché, and I hate even saying it — that we’re going to unlock the creativity of our people. But I’ll tell you this story. You tell me how to describe this, that [it] doesn’t sound cliché. I was once sent to one of my workshops. A 25-person team was sent by their company. It was the true multi-headed hydra of the most despised functions in corporate America. It was a joint finance and IT committee, tasked with creating a new finance IT system that would be a new global standard for how this giant corporation would do…

      Sonal: Oh, my God, it’s not even just the entities you’re describing, it’s the fact that there was a task, and a committee, and a standard. I mean, there’s just a lot of crazy stuff in there already.

      Eric: When I talk to startup-y and product people about the story, they start moaning and groaning before I even get to the — I haven’t even got to the setup, let alone the punchline, and they’re like “Oh, God, right, you know exactly…” And I was like, these people were not happy to be in this workshop. And they’ve been sent to this thing. And when I went there, I was like, “Okay, if you wanna do this, you gotta think like a startup. You’re gonna adopt a customer service mentality, and really, like — you got to understand that the people who use this product are your customers, even though they’re employees of the company.” And I thought I was gonna be burned alive. I mean, the looks I was getting from people were just like, “Who is this kid telling us what to do?” But we did the work, and we went through the method. Like I said, I think being able to derive lean startup from first principles is very helpful.

      You know, accepting people’s skepticism is natural, and being able to walk them through — look, be skeptical, but what is the experiment that could demonstrate. After three days working with this team, they were totally transformed, and they changed their plan from this — their original plan was, like, this huge committee, they basically were gonna spend 18 months gathering requirements. Hand those requirements off to all these implementation teams around the world, who would spend another 18 months doing the implementations.

      Sonal: Healthcare.gov approach basically.

      Eric: I mean, it’s just healthcare.gov all over again, right? It’s like, “Listen, first of all, one of my rules is, the laws of physics are required — everything else is optional.” So, the word “requirement” just doesn’t apply to, like, a giant worldwide focus group of random things that customers ask for. Those are not requirements, those are hypotheses. Those are guesses about what customers might want. So, instead of doing this big global thing that’s gonna take three years, there’s gonna be no accountability, right? Because the committee will disband, the things won’t be done correctly, there’ll be no productivity savings. We’ll be in the same IT finance mess we’re in now.

      They took a different approach. And they decided, on their own accord, to condense down to a five-person dedicated cross-functional team. So, no 25-person committee. They went to their customers, the P&L leaders of the different businesses in the company, and they made them an offer. They said, “Whoever says yes to this offer, the next day, our whole team is on a plane to your headquarters, wherever you are in the world. We are gonna sit with your people and build the software live, before your eyes, and we will show it to you every month, or every couple of weeks,” I don’t remember the sprint iteration cadence right now, “But we’ll show it to you periodically. And when you voluntarily decide to adopt it, you adopt it — so, no corporate mandates. We will show that it’s better than what you have now, and we will not leave. We will keep iterating the software for as long as it takes to prove to you, P&L leader, that you have an actual productivity improvement.” So, not “we met the requirements and now good luck,” but we will keep measuring how much work is actually done in this function — and it’s kind of complicated what the thing was, but we had the metrics to prove that it’s a good idea. And then and only then will we take it to a second P&L, a third P&L, and then eventually scale it up to the whole company.

      Sonal: Right. This goes back to your point about the methodology for scaling, and that being one of the biggest challenges.

      Eric: Yeah, exactly right. One of my — someone told me that the easy way to remember it is — think big, start small, scale fast. It’s, like, that’s really that is — so, like, prove that it works at scale X, then prove that it works at scale 2X, and just repeat until you have the whole thing. These guys transformed into honest to God entrepreneurs, every bit as enthusiastic and creative as the people I meet, you know, in Soma every day.

      Sonal: Right. So, even though it is really cheesy, I agree, to say like, “Unlocking human potential and creativity,” I do think it’s a really important argument for a future of a world — because when we talk about everything becoming software — like, companies changing, everything changing. For a world where more things are getting automated. And being able to really have something to contribute as a human being with judgment and creativity, something you can’t codify into a program.

      Eric: The debate over robots stealing all the jobs and everything, like, has baked into both sides of the disagreement. This fundamental premise that work is boring.

      Sonal: Yeah.

      Eric: And so, like, should we let robots do the boring work, or should we let humans do the boring work? And it’s like, no, no, no, that’s just evil. I cannot buy into that idea at all. First of all, work that is monotonous and routine should be automated, because every human being has a right to use their creativity in their job. This is a lesson going all the way back to the Toyota Production System of years ago. Even on the factory — like, the canonical job that was supposed to be — all creativity sapped out by Fred Taylor back in the days, right? Someone just doing the repetitive, you know, stressful work on the line.

      Even there, human creativity can work to our advantage. How much more so in knowledge work and in management, and all these kinds of systems we’re talking about. So, to me, this is saying these companies have locked up a massive amount of human potential. I think the scale of this, we are only — it’s hard even to fathom, because these companies are so large, and so many of the people are trapped in systems that prevent them from exercising their independent judgment and creativity. I mean, that’s just — like, people debate whether that could be changed or it’s a law of nature, but it’s a fact that is happening today. And what I have seen is that you can change it. It’s hard not to sound utopian about it, but I really think it’s going to have a profound impact.

      Michael: I just wanna ask — the distinction, though, between — does that mean everyone needs to be an entrepreneur or entrepreneurial?

      Eric: Yeah, that’s a great question. It’s a source of great confusion, because the companies that have adopted this system formally. They’re usually, like — even lean manufacturing. The Toyota Production System, it’s not called lean manufacturing, it’s called the Toyota Production System. Every company makes it their own. So, like, at GE, they have this program called FastWorks, which is their version of lean startup. At Intuit, they have a program called Design for Delight. Everybody has their own version of it. And one of the questions they get all the time is, “Well, does this system apply only to special projects, or does it apply to everyone?” And every company has had to deal with this duality, where they say, “Actually, there’s two versions of this. There is the version for, like — when you wanna make a big bet — disruptive new product. And I think we’re talking about as a startup, as an atomic unit of work. It’s like one of the reasons why Amazon is so effective, they have the two-pizza team rule, and they can say, “That’s a good idea. Let’s throw a startup at it.”

      Sonal: Right. Just to clarify that two-pizza team rule being that the team should be big enough in size to only be fed with two pizzas.

      Eric: No bigger than you can be feed with two pizzas

      Sonal: Exactly.

      Eric: It’s a basic way of saying small teams, small teams can try things. So, they have like a — it’s a tool in the management toolbox to throw a team at something and not let it turn into a big sprawling committee, but keep it focused, keep the people cross-functional and dedicated. No multitasking, no passing work between silos. That is the death of innovation. But then you also have, like — at GE, they call it FastWorks Everyday. And they say, “Look, no matter what work you’re doing, even down to…” They always give this example of, you’re preparing a PowerPoint presentation for a meeting. Even that very simple task, you can ask yourself, “Who is the customer for this task? And is there a minimum viable product version? Is there a way to test and experiment? How do I really know that this work is valuable?” And the number of people in corporate America who do work where I asked them this question, I say, “Listen, what is the evidence? How do you know that the work you do every day matters to anybody except your boss?” I used to think people — everyone would just be like, “Oh, of course, I know.” And a number of people who are like, “I don’t know, I just assume.”

      Sonal: We’re just kind of moving along like zombies in the workplace.

      Eric: And you’re like, “What kind of job is that? Like, how are you going to feel at night when you go home, saying, ‘Gosh, I hope I accomplished something today?’” Versus now, like, imagine a world in which everybody knows it in their bones, because they had the scientific rigor to always be testing. And when you see that, I mean, I’ve seen the before and after photos of the people who’ve gone through that transformation, and it’s truly powerful. Even in places, you know, I was thinking about like a factory where we did this transformation. And we’re talking to, like, the union reps for the people in the factory, and to see them go through the transformation. Even places where I think we have a prejudice, they’re like — like, in certain cultures, certain places, certain functions, those people would never get it. I’ve seen it everywhere that you can see it, and it’s a wonderful thing to see.

      Implementing lean startup

      Sonal: Everybody gets it. So, I wanna wrap up and then, kind of, revisiting the question of, how do people adopt this amazing movement and mindset — as we’ve said, it’s important — and the tools that come with it — without veering into cult territory. Where they start holding onto, like, the letter of the rule, versus the principle behind the rule. Because that seems to be a phenomenon that happens with anything. But also we’ve observed [it] happening with — even with things like lean startup. So, how do you sort of…

      Eric: Oh, certainly. I mean, I hear — many of my VC friends complain, like, about lean washing, they call it. And it’s like the same old crappy…

      Sonal: Oh, lean washing. That’s so funny, it’s like green washing.

      Eric: …the same old crappy venture pitch, but now it’s lean. It’s, like, lean crappy venture pitch. And I’m saying, “Look, first of all, do not, please — if you’re listening to this, if you’re trying this at home — do not use the terminology to, like, dress up your dumb idea,” okay? In fact, I don’t care if you use the terminology at all. In fact, I have a secret product managers, like, meetup group that I do for one of the big tech companies, where we get together — it’s secret so I can’t name the group…

      Michael: What’s the name of the group?

      Eric: Yeah, I can’t tell you. I can’t even name the company. Yeah, because they work in a company that has a real strong “not invented here,” culture, and lean startup is forbidden. So, if they talk about minimum viable product, or pivots — like, inside the company is just like, “Do not want to hear about it.” So, what they have figured out is, like, they have to create a company-specific vocabulary version of it. So, we meet periodically to talk about, like, “How do we get people to adopt these concepts without…”

      Sonal: It’s like putting code names around it.

      Eric: Yeah, because, like, to be honest, what matters is that you have a precise and clear language that you can communicate with your co-workers about. I don’t care if you use my language or somebody else’s, as long as it has a rigorous foundation. And the people who argue about — like, I periodically get a phone call from someone and they’re like, “So and so is blogging a bad thing about lean startup, like, they’re using it wrong. You need to make them stop.” And I’m like, “I’m not the Pope. I can’t excommunicate anybody. Like, what power do you think I have to make somebody stop?” Like, that kind of, like, inside baseball…

      Michael: “Hold on, let me get the stone tablets.”

      Eric: Yeah, it’s like, chisel an extra — you know, and it’s funny, because I said in a recent talk, I was like, “Listen, lean startup stands for what works. So, if I said something wrong — if we discover ourselves as a movement — we use our own scientific process to discover new things. They obsolete the old things. And we should always be getting better, and what we wrote five years ago should always seem a little bit creaky, or we’re not learning.”

      Sonal: To close it out then, let’s just take the two most popular terms, which are also often specifically overused — and I think, therefore, have a lot of misunderstandings around them. Pivot and MVP. And I’d love to start with MVP, or minimum viable product. Because one thing that I’ve seen on the other side is that people sometimes mistake doing an MVPm when sometimes the big visionary ideas require you to over-investm and not under-resource a minimum viable product. So you, actually — sometimes you don’t wanna— so, I’d like you to talk about that.

      Eric: Yeah, Oh, I’m happy to talk about that. It’s really interesting, because people really love the bumper sticker version of lean startup. Okay, there’s certain concepts that just people really gravitate to. Pivot, MVP, continuous deployment, you know, and some of the, like, lean buzzwords are people all excited about. But if you actually read the book — which I think a lot of people who are doing this buzzword Bingo haven’t actually done — if you read the book, the vast majority of it — the bulk of it by weight is not about, you know, this jargon, but it’s about the management system. It’s about the innovation accounting. It’s about the math that underlies this way of working. And you know, I get that accounting doesn’t make a good bumper sticker, and we can’t cram the math into — so, I understand why the slogans get the attention. That makes sense to me.

      But one of the problems with that is people who have a kind of a flip understanding don’t really have the context to do it correctly. And minimum viable product, or MVP, that is a pretty common problem. The core mistake I think people make is they think that it’s a minimum product, right? Like, if you’re trying to do something small in this world, you don’t need to do an MVP, you just do it. Like, if the thing itself is cheap and easy and has an obvious application, and…

      Sonal: Especially if it’s software.

      Eric: Yeah, you don’t need to test it, you don’t need to experiment, just ship it. It’s only people who have a consequential vision that this is a good idea for or is necessary for it all. In fact, the first part of my book is called “Vision.” Because people think like, oh, taking a scientific approach to innovation means taking the creativity out of it. Which I think is very insulting to our friends who are scientists. I’m sorry, I think science is one of humanity’s most creative pursuits. And someone once said to me, “If you could turn entrepreneurship into a science, then everybody could do it, and that wouldn’t be good.” And I was like, “First of all, science is a science, and very few people are good at it, okay?” It’s actually, like, it’s really hard. With all due respect, like, some of the smartest people who’ve ever lived have been our scientists — and not just smartest — our most creative people. So, science is not just turning a spreadsheet, right? It really is having those leaps of insight to form good hypotheses in the first place and that is what — and all MVP is saying is, take those hypotheses and put them to the test in a cost-effective way, so that we can sustain our investment in the vision over time. 

      Some visionaries in this world are independently wealthy, and can sustain investment in their projects for as long as they want. Some have the knack for raising money, just on the strength of their personality, and some can even take a company public and resist the temptations of Wall Street and just do what they — I mean, there are some people who have that ability to sustain their vision indefinitely, but most people do not. Most people, even the ones that have very, very powerful visions, struggle with, “How do I command resources? How do I get resources put to me?” And they feel, frankly, a lot of resentment about the just huge amount of bullshit that is required to do what we call success theater — putting on a show about how good your thing is doing, so that you keep attracting investment. And I think every ounce of energy we force visionaries to put into success theater is an ounce of energy they didn’t put into making their vision a reality, they didn’t put into serving customers.

      Sonal: Just making it happen. Right.

      Eric: So, I think of lean startup, and MVPs in particular, as a way to demonstrate progress towards the vision to sustain that interest overtime during the long flat part of the hockey stick, when the vanity metrics are really low and there isn’t instantaneous overnight success.

      Sonal: So then, the last word — pivot — completely overused. I mean, it’s actually just, God, we waited so long to even say it on this podcast.

      Eric: Yeah, I mean, I have to, like — I’m a little bit embarrassed now almost how much it’s become an overused buzzword. And I saw this — there’s this cartoon, you can Google it. There’s a cartoon in the New Yorker magazine a little while ago, where there’s a man and a woman sitting in a café, and she says, “I’m not leaving you, I’m pivoting to another man.” I just thought, “What have I done? I’m sorry.”

      Michael: No. Well, it is — I mean, you called it a movement when we were talking about it, and, I mean, that’s kind of part of it, right? I’m sure people have “pivot” tattooed. That’d be a weird tattoo to get.

      Eric: I’ve seen some pretty odd things.

      Michael: Yeah, but it is part of this, kind of, I don’t know — language and culture and fascination with what you’ve done with lean startup.

      Eric: Yeah. You know, first of all, I don’t apologize for the fact that pivot is a very useful concept for startups. In fact, you can go back and read stuff that was written about startups before that word came into the common vocabulary. And people struggle to explain this weird phenomenon, which is — it seems like the great entrepreneurs persevered through everything, and they, you know, stuck to the vision no matter what. And yet, they also were super flexible about certain details. So, like, that’s odd. Like, “What do you do?” And so, like — for the people who are new to the concept, the right definition of a pivot is a change in strategy without a change in vision, right? So, the vision is our true north, our destination. But the specific strategy we’re gonna be like, “What is the business model? What kind of product is it? Is it software? Is it hardware? Is it a device? Is it service?”

      I think about, you know, the Google search appliance and the pivot to AdWords, right? That didn’t give up on the vision of organizing the world’s information, but they said, “That’s a dumb business model, and this is a good business model.” So, like, that’s okay. But if someone had said, like, “Let’s get out of the search and information business and just sell cars,” you know, they would have been like, “Oh, that’s not a pivot. That’s abandoning our vision.” Although, of course, now they are gonna sell cars, so the vision expands as you have success.

      Sonal: Actually, I think it is still connected to all the world’s information, right? Especially as cars become moving computers.

      Eric: Oh, yeah, well, exactly. So, you know, vision is personal, and it’s deep in the minds and the souls of the founders and eventually into the whole company in its DNA. So, from the outside, it can be a little bit hard to understand, like, what really is the vision and what are the incidental bits? Like, I think people, you know, certainly would have thought, at a certain point, that Netflix is all about sending you DVDs by mail. And when they first — I remember when people — it was very controversial when they started to become an online streaming service that would actually have fewer movies and less options. It seemed like it was getting worse. Like, it’s not an abdication of the vision and then…

      Sonal: And now to programming their own content.

      Eric: I mean, yeah, so you never know where — you know, you have to be flexible about the specific strategy, but you have to be willing to invest in and stay true to the vision. So, that really is why the concept of pivot is so important.

      Sonal: That was the origin of the word.

      Eric: And the reason it’s such a critical part of lean startup is, if we can get to the moment of pivoting sooner, cheaper, faster, it’s like magically extending the runway of the startup without raising more money, and that’s why it’s such a powerful idea.

      Michael: It’s a movement. It’s not a cult. It’s gathering momentum, you know, outside of Silicon Valley and beyond. And we’ll try and extract the information on your secret product management group later.

      Sonal: I’m gonna start, like, tailing you and driving following you everywhere to see where that goes.

      Michael: Thanks so much for coming. We’ll look forward to the next book.

      Sonal: Thank you, Eric.

      Eric: Thank you very much.

      • Eric Ries

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Michael Copeland

      Holy Non Sequiturs, Batman! What Disruption Theory Is and Isn’t

      Sonal Chokshi and Michael Raynor

      Disruption is such an overused buzzword. But the word itself does have meaning: As defined by the Oxford and Merriam-Webster dictionaries, it is a “disturbance…that interrupts an event, activity, or process” and that causes something “to be unable to continue in the normal way.” It’s also the name for an influential theory about innovation first coined by Clayton Christensen in a 1995 article and later publicized through his 1997 book, The Innovator’s Dilemma.

      But that was nearly two decades ago! Not only has the concept been much misunderstood and mangled since then, surely it’s changed given the advent of new tech and business models today? Is it still relevant, given cases that seemingly defy the theory and its application? Are we at risk of overfitting this “verbally inflated” term to everything, and in doing so, are we missing what disruption theory really says — and doesn’t?

      Michael Raynor, co-author of the followup book on disruptive innovation with Christensen — and author of another book that later tested the predictive power of the theory — joins this episode of the a16z Podcast, in conversation with Sonal Chokshi, to answer these questions and more. He also hints at some nuggets from an upcoming article in Harvard Business Review with Christensen and others that addresses the latest formulations of this theory of innovation.

      Show Notes

      • What disruption theory is, and how it’s been misunderstood [0:34]
      • Popular examples of “disruption” that do not conform to the formal definition [10:23]
      • The difference between disruption and innovation [24:42]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal Chokshi, and today we have as a guest on the pod Michael Raynor, who is a director at Deloitte Services and a co-author on one of the seminal books on disruption, with Clayton Christensen, “The Innovator’s Solution.” Raynor also later wrote a book called, “The Innovator’s Manifesto,” which is one of the very few works out there that actually tests the predictive power of disruption theory. We invited Raynor on the podcast today, since he, Clayton Christensen, and Rory McDonald have a new paper coming out this December in Harvard Business Review, defining what disruption theory is and what it isn’t. Welcome, Michael.

      Michael: Thanks. It’s great to be here.

      Defining “disruption”

      Sonal: Thank you for coming. So, actually, why don’t we just jump right in, and let’s just start with talking about what disruption is. And I think it’s come top of mind, because there’s been a lot of articles written in the last year that, sort of, slam it. And, to be clear, I think people are slamming both the word, because it is very overused, and I think part of it is also slamming the theory, which is what we’re talking about here — disruption the theory.

      Michael: Yeah, I mean, as I read it, I’d say it’s even more than part of it. I mean, there have been a couple of critiques that have come out. There was a piece just over a year ago in the New Yorker that was very pointedly about Clayton and disruption theory. There’s a piece [that’s] come out recently in the Sloan Management Review that’s similarly, sort of, an attempt to look at the theory and say, “So, here’s where it goes too far, where it makes mistakes, where…” And I think that’s important, actually. I mean, that’s how we progress, right? Constantly saying we got it right the first time doesn’t take you anywhere new. Conceptually, at least, it’s easy to embrace that kind of give and take, that kind of discussion.

      Sonal: Yeah, no, I think that’s great. What was the gist of the critiques? I mean, I agree that conceptually, it’s important to have that kind of — but what were people, sort of, slamming about the theory?

      Michael: Yeah it’s — I guess, in the first instance, they were going after a phenomenon that I think is definitely something worth trying to, I’ll say, combat at the risk of overstating the case. Which is that the term “disruption” I think has come to be used far too frequently with far too little precision, and I think that’s unfortunate. I mean, in the first instance, disruption is a well-formed English word. You can look it up in the dictionary, right? But when people use it, especially when it comes to — in a business context, they forget what the word actually means. So, disruption means to hold up something, to slow it down, to interrupt an otherwise smooth and even flow. So, a disruptive student is not something that makes class better. That’s someone who makes class worse. A disruption in subway service doesn’t help you get where you want to go better or faster or cheaper.

      Sonal: Yeah, you’re right. It doesn’t have a positive connotation at all.

      Michael: It’s just a bad thing, but people that’s not what people say. People say, “Oh, you know, this company is disrupting things.” And that’s supposed to be good. So, when they say that, they’re invoking — whether they know it or not, they’re invoking the connotation that Clayton gave to the word, with disruptive innovation and disruption theory. But unfortunately, they’re using it, then, in a way that is, to my way of thinking at least, more often than not entirely disconnected from the specifics of what disruption theory actually describes. So, we’ve run into a circumstance where people use the word very frequently to describe all manner of phenomena. And the downside of that and this — and here’s why this matters — the downside of that is that as a consequence of that verbal inflation, we actually lose our grip on the power and the insights that disruption theory brings, and that would be a shame.

      Sonal: I totally agree. And I think that’s actually a great turn of phrase — verbal inflation — and one could argue it’s a disruption bubble. When Clay originally coined the phrase — used the word “disruption theory” — like, what was the intent behind the original meaning?

      Michael: So, his first book came out in ’97. It was called “The Innovator’s Dilemma,” and it started slow. It came out in ’97, sold, you know — I’ll get these numbers precisely wrong, but close enough, close enough. Sold a few thousand copies over the next couple of years, and then it exploded in 1999. I think it was the January issue of — I think it was the cover of Forbes, and it had Clay with Andy Grove. Caption read, “Andy Grove’s big thinker.” And Clay is 6′8″ for those people who haven’t met him, and Andy Grove is probably, I don’t know, 5’6” so it was a clever turn of phrase.

      In any event, then the book took off. And what that book described is a particular class of phenomena whereby companies are able — small, under-resourced startups, very often — are able successfully to enter markets that are dominated by well-managed incumbents. And so, that was a puzzle. How is that possible? How does this small, you know, scrappy, little upstart — how is it able to successfully overturn a successful incumbent? And so, Clay chose the word “disruption” to describe that phenomenon. And it captures half of what’s going on, right? The disruption is to the incumbent, and it describes a very particular pathway by which a startup and a new entrant, more generally, is able to enter an established market.

      Sonal: Right, and if I remember correctly and, actually, I remember learning this from you a few years ago. It’s, sort of, the startup comes, in traditional disruption theory, from the lower end of the market, usually with either lesser features, or just reaching a niche customer set that no one is otherwise reaching. And then the other part of it, the second part of it, I remember, was really key — is that there’s some kind of accelerator through technology that then drives them to be able to go upmarket.

      Michael: So, this notion of what you call an accelerator, what I refer to as an enabling technology — what Clayton has referred to as an extensible core. So, the fact that the language hasn’t quite settled down shows that this is a relatively new addition to the theory, but I think a critically important one is one that a lot of people have walked past and fundamentally ignored.

      Sonal: Right. Are they able to ignore it because they’re just, sort of, fitting the theory to everything? Or is it because you don’t actually require that accelerant in order to reach that sort of quote disruption?

      Michael: No, I think it’s a necessary condition. You don’t have that, you don’t have a disruption. So, you pointed to a couple of things, which is that a disruptor starts at the low end, a niche market. That’s a defining feature. There are really three necessary and sufficient conditions, right? The first one that we pointed to was where you start. So, first and foremost, disruption theory is a theory of customer dependence. You tell me who you’re selling to, and I’ll tell you whether you’re embarking on a potentially disruptive trajectory of innovation. So, disruptors start in segments of the market that incumbents aren’t motivated to fight for, or fundamentally don’t see. So, we refer to that as either the low end, or an entirely new market competing with non-consumption. So, that’s step one.

      Second is, you have to have a fundamentally different business model that allows you to serve profitably the niche that the incumbents don’t want. There’s a reason the incumbents don’t want to serve those niche markets — because they can’t do it profitably enough, right? And so, you have to come up with a way to serve those segments profitably. If they lose money there, and you say, “Well, I’ll go lose money there too,” that’s not gonna be a disruption. That’s just losing money. So you have to have a different way of serving those segments. Then the last piece is this enabling technology. It’s something that allows you to now take that same business model and begin to serve the mainstream markets that the incumbents do care about. But now it’s too late, because the incumbents can’t respond, because you have broken the trade-offs that they were depending on. The trade-offs that made it impossible for them to serve the low end, you have now broken.

      Sonal: So then what has changed today? Because one of the observations that I have, and some of this is definitely anecdotal, is that there seems to be — it seems to be happening a lot faster, for one thing, because of “software eating the world.” There are arguably new patterns. I mean, I’d love to hear your thoughts on this. 

      Michael: So, I guess what I’d say is that when it comes to things happening faster, that speaks to the rate of change in the underlying enabling technology. So, if you look at disruption in the steel industry, right, how long did that take? So Nucor is the archetypal disruptor in the steel business. Where did it start? It started with rebar, which is a low-volume, low-margin segment of the steel business that incumbent steel makers were not motivated to defend. Nucor built a fundamentally different business around the mini-mill, and then it took 43 years for Nucor to become the same size as some of the largest integrated mills in the U.S.

      So, why did it take 43 years? Well, it took 43 years. What was Nucor’s enabling technology? Well, it was electric arc furnaces and continuous casting. And those big iron — both, literally and metaphorically — those big iron technologies improve relatively slowly. They improve on a mechanical clock speed, and so it took 43 years. And then you look at disruptions in the tech space, and you say, “Well, what about the personal computer?” So, the personal computer is clearly a disruption to mini computers and mainframes for all the same reasons. Started as toys, sold to hobbyists that couldn’t do anything. What was the enabling technology there? Well, it was the microprocessor, and the microprocessor — that gets better pretty quick. The difference is not the underlying phenomenon, nor the theory — it’s an empirical observation. Which is, how fast does the enabling technology get better? That will tell you how quickly it will break the trade-offs that preclude it from serving mainstream markets.

      Sonal: Actually, I’ve actually heard from Alvy Ray Smith, the co-founder of Pixar, that they used that exact formula in their head to actually then map out how they would intentionally disrupt the making of animated films.

      Michael: Sure.

      Sonal: Because they were able to actually use it to, like, almost predictive power in a sense.

      Michael: Part of the reason that I find disruption theory so powerful is that now when people say, “Well, this is completely different because it’s so much faster,” I’m like, “Actually, no, it’s not completely different. It is a quantitatively different outcome, but it is qualitatively the same phenomenon.” We can use the same theoretical toolkit to understand what’s happening. The specifics are different — that’s why we play the game — but we can use the same theory to understand and, to your point, predict and maybe even control.

      What disruption theory is not

      Sonal: So, speaking of prediction, you know, again — you’re one of the few people who actually applied and studied the predictive power of disruption theory. What are some of your high-level findings from that work?

      Michael: “The Innovator’s Manifesto” came out in 2011, and it was an attempt to do, as you say, to actually use the theory to predict outcomes. And it’s tricky. It was a lab experiment, and it was done using MBA students largely. I’ve had a chance to replicate it using executives now, and have achieved essentially the same results.

      Sonal: Oh, that’s good to hear so that reproducibility… <crosstalk>

      Michael: Right, exactly, yeah. Although the executives weren’t too thrilled to hear that they weren’t doing any better than the MBAs but, you know, sometimes the truth hurts. And we did that using a portfolio of businesses that had been launched by Intel over the years. And it was a randomized, double-blind, you know, study to say, “Right, we went to the MBAs and gave them a bunch of business cases and said ‘Pick winners and losers’ and then we taught them disruption theory, and we said, ‘Now, try it again.'” And I’m glossing over all the details that make the findings, I hope, believable. But what we found is that the users of disruption theory improved their accuracy by up to 50%. That said, in absolute terms, we have to be modest. Their success rate was around 10% at picking winners, and it was about 15% picking winners with disruption theory, because it’s a big, noisy world.

      Sonal: And you also have a sample set that’s an internally captive VC arm, essentially, a venturing arm that’s inside a company.

      Michael: Right. All kinds of delimitations. You know, my experiment, you know, like, every other has its share of imperfections. But to your point, it was an attempt to actually try and take seriously the notion that the theory can be used to predict, and I found the findings encouraging.

      Sonal: We do have a tendency. You know, I think the predictive power matters, because there are entire businesses built upon this theory, and some of them which have become incredibly successful. One question we have is — there’s a tendency to, kind of, equate technology means disruption. Just sort of, you know, to over-apply and overfit the phrase to everything. So, what is disruption theory not? Like, what’s not disruption theory then, to help people kind of understand what it is?

      Michael: That’s a great question. In fact, both Clayton and I and another professor at HBS named Rory McDonald have a piece coming out in the December issue of the Harvard Business Review that tackles that.

      Sonal: Oh, give us the early preview.

      Michael: Yeah, exactly.

      Sonal: That’s why I want you to tell us all your secrets.

      Michael: Sure. Well, I’ll give you an example. It’s — and I’ve asked this question at various conferences and workshops I’ve been part of — I ask for a show of hands. How many people think Uber is disruptive? Every hand in the room goes up.

      Sonal: Ours included. One of ours went up.

      Michael: Yeah, exactly. And it’s not. In fact, it’s the…

      Sonal: So why?

      Michael: Well, let’s review the theory, right? I mean, disruption theory is, first and foremost, a theory of customer dependence. Whom are you selling to? So, whom did Uber sell to? Was it selling to a niche of the market, the low end of the taxi market that established taxi simply couldn’t be bothered to serve? Was it selling to people who found hailing a cab and paying for it so inconvenient and so expensive that they just had never used cabs before? No.

      Sonal: So, it’s not capturing that different consumer market, right.

      Michael: They were going after, and continue, for a large part of the business, to go after folks who want a cheaper, more convenient, cleaner, nicer cab ride, right? There was an article in Businessweek — again, I think I’m remembering this largely correctly — and it was stating that Uber had gone from — again, close enough, close enough — 350,000 rides a month in Manhattan to 3 million rides a month in Manhattan. And over that same period of time, what do you think the drop off in yellow cab rides was? Son of a gun, about 3 million rides a month.

      Sonal: So, ride-sharing is clearly a huge market, but you’re saying that because they’re competing with the same exact customers as the taxi industry, it doesn’t count on that one criterion, so far, as disruption.

      Michael: Well, exactly. And so, remember, what disruption describes is a pathway — a particular way in which a small under-resourced entrant can succeed against well-managed, dominant incumbents. So, it’s a pathway. It’s not a description of your impact on the established market, which is how people have tended to use it. Say, “Oh, Uber is disruptive because it’s turned the industry upside.” Well, it has revolutionized the industry. It has had a huge impact on the industry. It is not…

      Sonal: But it’s not technically disruption.

      Michael: Well, but you say that as if somehow it were a minor distinction.

      Sonal: No, right.

      Michael: Yes, it’s not technically — it’s not disruption and that matters, because if we think it’s disruptive, then other folks who want to pursue a disruptive strategy will think, “Well, I need to do what Uber did…” What did Uber do? Uber did something that, to my mind, at least, is a fairly long-odds proposition. Which is, they just built a better mousetrap.

      Sonal: So, that’s interesting, because one of the theses that one of our partners put forth a couple of years ago is something called the full-stack startup, which — you know, he sometimes jokes about how he regrets even calling it that, because it’s sort of like an analogy…

      Michael: You should talk to Clay about regretting having called disruptive technology.

      Sonal: Oh, I know, right. Clay is probably the one who has a lot of regrets around those things. But the way Chris Dixon articulates the thesis is that, in the past, companies like Lyft and Uber would have tried to build software and then sell it to the taxi industry. But there weren’t even people in the industry who could even have the skill set, let alone to appreciate the software — to evaluate that software and actually say, “Okay, this is what’s going to help us with the problem we have.” Nor were they incented to solve for that problem. 

      And so, he argued that instead of trying to go down that path, there’s been a new wave of startups that’s actually been able to “disrupt” — and, yes, I agree, this is not in the technical form of disrupt, but now I’m using it as more of a descriptive adjective — that they’re able to overturn and shake up, so to speak, the taxi industry, because they built something full-stack. Like, from end to end, so they can control the entire experience. And by doing so, they essentially stopped trying to sell their software to the taxi industry and just built an alternative. Like, to your point, a better mousetrap.

      Sonal: Yes. They built a better mousetrap, and as Emerson said, the world beat a path to their door. In fact, they probably took an Uber to their door.

      Michael: Right. But I do wanna just protest for a second, Michael, because I’m having a really hard time letting go of this belief, and you’re gonna have to convince me a little harder.

      Michael: You and everybody else, I’m sure.

      Sonal: Right, I am. I’m fighting it. But the reason is because it does feel that — what if it means that disruption theory could be adapted for the software world?

      Michael: Well, no, it’s a different phenomenon. And, now, when I say adapted for that — we kind of talked and touched on that earlier. If we are describing a phenomenon in which software is the enabling technology for an entrant on a disruptive path, we’re describing something that starts over on the fringes and works its way into the mainstream, with a fundamentally different business model that is powered by — and this is key — in fundamental improvements in the software over time. All of those things have to be there. And it’s important to understand that, because it feeds into the choices that you make as a manager along the way. How do you deploy resources? What R&D strategy do you follow? What customer segment do you target?

      Sonal: Right. Do you build self-driving cars or not?

      Michael: These are all things — so it’s important, I think, to underline that — I’m not, sort of, being picky. At least, I hope I’m not being picky here.

      Sonal: No, it’s great. That’s why we wanna have this discussion.

      Michael: The phenomenon you describe — that one is describing and the meanings that one attributes to these words are critically important, because they determine the choices we make. The way we use the words is critically important because, ultimately, what happens is that everything is disruptive. And when everything belongs to a category, then the category is useless.

      Sonal: So then, how has the theory changed? You are already talking about what disruption theory isn’t. Are there any other examples along those lines? And I’d love to hear your thoughts about what has, sort of, updated around here.

      Michael: So, I guess there’s a couple of things that I’ve observed that people have — that threatened to lead folks astray. One is, sort of, it happened fast, you know, it’s a big bang disruption. Well, no, we actually don’t need that because we have this concept of the enabling technology, and it’s the rate of improvement in the enabling technology that determines whether or not it’s a disruption. Now, there may be complete transformations of an industry that happened very quickly, for reasons other than the disruptive entry of the startups — and that’s fine. But then we need to be clear, that’s a different phenomenon.

      Sonal: Even if the outcome may be the same, actually, in some cases.

      Michael: Absolutely. These are, you know, to use a medical analogy, these are different conditions, and you need to get the diagnosis right. So, that’s fine. The other thing that I think leads people astray is the notion of — and you hear this, you know, kind of, top-down disruption — and people will point at Tesla, and they’ll say, “Tesla is a — they’re disrupting the car industry, but they’re doing it from the top.”

      Sonal: So, they’re not disruptive.

      Michael: They’re not disruptive at all.

      Sonal: Okay, so let’s talk about why.

      Michael: Well, same reasons, right? Was Tesla targeting a small, unprofitable, unattractive segment of the car market that was of no interest to incumbent car companies? No, they’re targeting people willing to spend 100 grand on a car, which is very interesting and important to companies like Mercedes and BMW and Lexus and, and, and…

      Sonal: Right. But it was an underserved market, in the sense of — those folks were not having a car that has software at its center.

      Michael: No, no. They weren’t — you think they didn’t have a car?

      Sonal: Of course, they had a car.

      Michael: Well, then they were consumers.

      Sonal: No. Okay, I’m gonna fight this one too. Again, this is different because, yes, you’re right. They would have bought another car. They would have been — they are the typical segment for other car companies, so that’s not a new market in that sense, but they weren’t having — their needs were not being met.

      Michael: They were underserved?

      Sonal: They were underserved.

      Michael: Absolutely. And that’s the sustaining innovation. Disruptive innovations target overserved customers.

      Sonal: There we go.

      Michael: Customers for which established solutions are too good, too expensive, inaccessible.

      Sonal: Okay. So that’s another precision thing that helps us define what disruption is and isn’t.

      Michael: So, Tesla goes after a critically important segment of the market, and shows up with, you know, its own version of a better mousetrap and appeals to those they’re willing to buy, and away we go. And so, in fact, if I were to point to somebody that explains the path that Tesla appears to be following, I point to Jeff Moore in “Crossing the Chasm.” As I read it, the way you cross the chasm is that you find very demanding customers, and you create a highly effective solution that solves their problems really well. And then, basically, you ride a cost-reduction curve into the mainstream, right? So, you find the really demanding, early adopters. In a sense, it’s an adaptation of Everett Rogers’s diffusion theory.

      Sonal: That’s actually where the whole “Crossing the Chasm” thing actually was hinged on?

      Michael: Yeah, so you find those really demanding early adopters, you solve their problem because they’re demanding, they’re willing to pay, you use the profits that you generate from serving those high-demand, very profitable, early adopter customers, and then there’s a lot of things you have to do in order to cross the chasm into the mainstream. That’s completely different from what disruption describes. Disruption describes a very different path from the fringe to the mainstream.

      Disruption vs. innovation

      Sonal: So, so far, we have Uber and Tesla, which a lot of people — including me, apparently — thought were disruptive and really aren’t. So, are there any examples where the company is actually disruptive but no one really knows it is?

      Michael: Well, that I can’t speak to, but you’ve probably heard of Theranos.

      Sonal: Of course.

      Michael: So, I would look at that one, and here — I’ll probably run out of facts sooner than I should. My understanding there is that they’ve created a whole series of blood tests that are able to give a high level of accuracy at very low expense and very low inconvenience. The way I think about that is that it’s an innovation, because it has broken trade-offs. And something that I think gets in the way, is that when we think about disruptive innovation, we can’t separate disruptive innovation from any other type of innovation, because we don’t have the larger class defined. So, an innovation for me is anything that breaks a constraint.

      Sonal: I love that definition, by the way. I’ve actually stolen and used that definition for years since reading your book. I just want to tell people publicly — that that wasn’t my idea. I want to just confess.

      Michael: Coming clean after all these years.

      Sonal: I actually did credit you, in fairness, but I am gonna say that that is, I think, by far the best definition of innovation I’ve ever heard. I really mean that.

      Michael: Well, thank you. We can stop here. So an innovation is anything that breaks a constraint and disruptive innovation is a particular path, right, from not being able to break those constraints to having broken them in the mainstream markets. So, when I look at Theranos, my understanding of it is that their solution right now is kind of — is more for less. They’re having difficulty, I think, finding adoption in, you know, mainstream hospital labs, and so they’re actually finding their foothold, their first commercial applications in clinics and drugstores and relative — essentially, if you will, on the fringes of the core mainstream blood testing market. So, you have something that is — that has broken certain constraints and is following a path from the fringe to the mainstream. What I don’t know enough about is whether there’s, at that core, the enabling technology that is going to allow the “Theranos solution” to, in scare quotes here, improve to the point that it can penetrate mainstream markets.

      Sonal: Got it.

      Michael: And this is important, because if it’s there already, right, if it’s already more than good enough, right, for those applications, then what we have is not bona fide disruption, what we have is a marketing strategy.

      Sonal: Right?

      Michael: The need to start at the fringe and move to the middle. And so, the kinds of things they get caught up is people say, “Well, it started small and got big.” Well, that doesn’t make you disruptive. That just means you started small and got big. Almost nothing big starts big.

      Sonal: That’s actually a good point.

      Michael: Right? And so all of these other characteristics — when people say, “Well, it started small and got big and it revolutionized the industry, therefore, it’s disruptive.” Holy non sequiturs, Batman, none of those things have anything to do with whether or not you’re following a disruptive path. Disruption can be used very precisely, and it describes an important class of phenomena, but it’s not a theory of everything.

      Sonal: Got it. So, let’s actually then take on the elephant in the room and talk about — and, again, I don’t wanna make this about Clay. I know we both have immense respect for him. But people often argue that he was wrong about the iPhone. And, I mean, I’ve made the argument. I know others have made this argument, that it was a category error. That he just got the category wrong for what he actually thought it should be when it was something else. What’s your, sort of, take on, sort of, why that did actually apply or didn’t apply in that case?

      Michael: Sure. So, I think — and this is a subtle but critically important distinction to make between what I’ll call the cross-sectional problem and the longitudinal problem, right? So, Apple showed up with the iPhone in the mobile phone market with a better mousetrap. Apple did not enter the smartphone market disruptively. And, again, why do we say that? Let’s see. Whom are they trying to sell the iPhone to? People who had phones, right? People who wanted a better phone, right? People who wanted a phone that could do other stuff, right? It’s not as though they were appealing to a niche market…

      Sonal: For underserved customers, right, exactly.

      Michael: …that established phone makers said, all of them, “Apple can have them. We don’t really want those customers anyway. Who needs 18 million more customers?” Of course, right? So, they were selling to customers. And Clay, I think, was absolutely correct in the way in which he applied the theory. He said, “Look, the data say pretty clearly that if you kind of walk into a bar and punch the biggest guy there, you’re in for a fight, and chances are you’re gonna lose.” That’s not what happened, right? So Apple, in my view, beat the odds in the way that if Tesla is ultimately successful, Tesla will have beaten the odds, and that’s fine, right? That is a class of phenomenon that needs a theory to explain it. How is it that some companies enter well-established markets and prevail when that’s such a long-odds proposition?

      Sonal: As you know, like — folks like Ben Thompson, John Gruber, myself — others have made the argument that it was disruption, but because it was a disruption to the PC industry.

      Michael: But that’s the longitudinal problem. The cross-sectional problem is how did they enter the smartphone market? They entered it with a sustaining innovation, and it worked. Good for them. Now, how did they realize growth out of that? Well, they were busy racing up the disruptive trajectory displacing the personal computer. Terrific. Every company is playing both games at the same time. They have to be winning the cross-sectional battle they’re in…

      Sonal: As well as gaining points for the long game, yeah.

      Michael: We’ll go back — you know, sometimes with the benefit of, you know, the perspective that history provides. <Hindsight.> I’m not reinterpreting — if you look at say, Xerox and personal copiers.

      Sonal: Right, we talked about this a few years ago.

      Michael: Absolutely. This is near and dear to your heart, I know. So, the early personal copiers, they had a cross-sectional battle to win themselves. They were competing with carbon paper and Gestetner machines. So, they were more expensive than those — so they had to be better, right? They had to win the cross-sectional strategic battle for the niche market that they wanted. Now, it was niche to Xerox, but it wasn’t a niche market to the folks who made carbon paper and Gestetner machines, so the personal copiers had to win that fight. Right? And then they followed the disruptive path into commercial applications for photocopying technology. And, by the way, you need a different toolkit to understand how to win that cross-sectional battle. That’s a strategy problem, right? Strategy is about the constraints you embrace. The innovation problem is about the constraints you break, and you need a different toolkit to understand that. And I think a very powerful tool in that toolkit is disruption theory, and there are other tools — diffusion theory, crossing the chasm, there are others.

      Sonal: So, another thing that people tend to equate when it comes to disruption — and this actually comes up in the case of the iPhone that we were just talking about — is that disruption equals money. Clearly, not all wildly successful products are disruptive. Is that true the other way around?

      Michael: Yes. So, if disruption were defined as being successful, it would be useless as a theory.

      Sonal: That’s a good point.

      Michael: Right? And so there are any number of efforts that have tried to follow disruptive paths that have ultimately failed. We’ll go back to the core research that led Clay to create or discover the theory, depending on how you think about these things, in disk drives. So, each subsequent generation of disk drives — you start out with, you know, the Winchester drives, and then the eight-inch drives, and then the five and a quarter and then the three and a half. And with each generation of disk drives, there was a ravenous horde of companies that were seeking to deliver that new generation of technology, all eager — and, in fact, quite ably — following the disruptive path. And guess what? Not all of them succeeded. Some did, some didn’t. Back to my earlier observation — they have to win the cross-sectional battle as well as the longitudinal one.

      Sonal: The longitudinal one, right.

      Michael: And disruption theory doesn’t say anything about that. That’s not a shortcoming of the theory. That’s not, as they say around here — that’s not a bug, that’s a feature. Right? Because it’s not a theory. Theories are powerful when they have boundaries, when you know what phenomenon they are used to describe. It’s like an antibiotic. If you take antibiotics when you got a cold, you’re actually doing yourself harm. And the same goes for any good theory, right? If you start applying it when it doesn’t apply, you’re highly — in fact, you are more likely to make the wrong decisions than if you just didn’t use it at all.

      Sonal: We have a lot of entrepreneurs in our audience, and I want to make sure that they — you know, that we’re not just talking theory, that there’s something concrete that we can do with this information. How do you resolve the tension between this — you know, if you’re focusing on the long game, the longitudinal battle, how do you, then, address, sort of, the cross-sectional reality that’s right in front of you?

      Michael: Yeah, so that I would put in a category of a strategy problem, right? How do you actually create a strategy? How do you embrace different trade-offs in a different way from your competition, so that you’re differentiated in a way that customers find valuable? The good news is that there, once again, there’s a long stream of both scholarship, theoretical and applied, that seeks to tackle that problem. I’ve tried to make my own contribution to that body of work as well. In 2013, my book, “The Three Rules” came out with my co-author Mumtaz Ahmed, and that was an attempt to try and unpack — what does it take to win in the here and now? When you face trade-offs, which trade-offs should you embrace, and how do you go about remaining committed to those choices over time?

      Sonal: How do people decide to make those trade-offs? Like, what should they know?

      Michael: Better before cheaper, revenue before cost — and there are no other rules, if you’ll forgive me.

      Sonal: That’s great.

      Michael: And it’s intended to look at, kind of, the three core questions that I think define any business. In the first instance, how do you create value for your customers? And there’s basically two ways you can do that, right? You can provide superior value or you can provide lower price. And we’ve concluded that companies that deliver exceptional profitability over time focus systematically on better before cheaper. The second question is, how do you capture value for yourself in the form of profits? And here, the arithmetic of profitability is pretty straightforward, right? It’s just revenue minus cost. Guess what? Companies that deliver superior profitability focus on revenue before cost. And then, finally, what do you change when everything around you changes? And the answer is anything, except those first two rules.

      Sonal: Oh, that’s great.

      Michael: Which is why the third rule is, there are no others. So, those are rules that I think — they pass the test of being falsifiable, right? If I’d said the rules were cheaper before better, I wouldn’t be talking nonsense. There are people who actually think price-based competition is extraordinarily powerful. Look at the big discounters in any industry. What we found is the data point in the other direction. If I told you that, you know, being a cost leader is key to superior profitability, you probably think, “Yeah, that makes sense.” And it does make sense. It just happens not to be true, which is that systematically, over the long term, companies that focus on superior revenue, either through higher unit price or higher total unit volume, are more likely to deliver superior profitability than companies that focus on cost leadership.

      Sonal: Yeah, I mean, just one last point on this. I think this is where the studies do get a little tricky, because we’re looking at larger data sets, but every success story — and I admit that there’s definitely a survivor bias when I make this claim I’m about to make — there’s an outlier of success that always just proves every theory. I’m thinking of Amazon, for example.

      Michael: No, of course, which is why — no, no, no question, which is why it’s called “The Three Rules,” not the three laws. And here’s what we think the rules are good for — which is that some folks may be of a mind that look, “You can collect the data, analyze the data, and come up with the answer.” The data are always ambiguous, right? What data mean is as much a function of what we impose on them as what they say to us.

      Sonal: That’s right. Exactly.

      Michael: And so if you can’t be bias-free, because you can’t, the best you can hope for, perhaps, is to have the right bias, right? Play house odds, if you will. So, when we look at it, we say, “Look, the bias should be better before cheaper, revenue before cost.” If the data convince you otherwise, then you should go in the other direction. We’re not gonna say, “Well, I’m just going to ignore reality and follow the rules.” That would be silly. It’s better before — and note, better before cheaper. It’s not better, not cheaper.

      Sonal: Right. You’re just saying how to prioritize and make those trade-offs, in that case.

      Michael: Well-played. I would agree. Exactly.

      Sonal: So, to wrap up a bit then let’s talk about — it’s a phrase that people here tease me about all the time — some “nuggety nuggets” that came out of your…

      Michael: I can see why they tease you about that.

      Sonal: I know, but I use it when I describe when we’re working on decks, like, “Where are the nuggety nuggets?” But, anyway, what are some of the nuggety nuggets coming out of your paper that you can share with us? Like, other things that, you know, are, kind of, some cool insights?

      Michael: “Innovator’s Dilemma” was the first popular expression of disruption theory. It wasn’t the first. In fact, Clay’s theory of disruption was really born in his doctoral thesis. But ’97 is a long time ago, and the first article that introduced it to a popular management article was in the Harvard Business Review in 1995. So, it’s actually 20 years since disruption theory was kind of introduced. It’s not like it has been frozen in amber for that 20-year period, and so it’s important to remember that. Some of what’s happened is that people have picked up “The Innovator’s Dilemma” and read that very carefully, and said, “Okay, I’m gonna go after this.” Which is — again, that’s how we learn. That’s how science progresses. Absolutely. But it makes a lot more sense to grab ahold of the latest formulation of the theory that takes advantage of everything that’s been learned over the last 20 years.

      Sonal: I know we have to read the December issue. We’ll wait. We’ll read it, but what are some of the other things you can share with us as an early preview?

      Michael: One is that I think disruption has come to be used in a way that people say they are not using in a technical sense, and they do not mean to invoke Clay Christiansen but, indeed, if you use disruption to mean something has revolutionized and improved outcomes, then that’s what you’re doing, right? Because the English word means to introduce chaos, not to introduce a new and better order. Right? So when we use disruption with an innovation connotation attached to it, then disruption theory comes along for the ride. And the bad news is that when that happens, we’re back to the verbal inflation problem. We actually lose the power that disruption theory has to offer, and that’s what concerns me, right? So, my hope is to kind of — that the December piece, in part, will begin to save disruption from its own popularity.

      Sonal: I love it. Saving disruption from itself. Well, Michael, thank you for joining the “a16z Podcast.” This has been a great conversation. I’m glad you disillusioned me. I’m gonna probably lose some sleep over some of those. No, I’m just joking. Not really. But, thank you.

      Michael: My pleasure.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      • Michael Raynor

      It’s Not What You Say, It’s How You Say It — When Language Meets Big Data

      Kieran Snyder

      When most people think of big data they think of numbers, but it turns out that a lot of big data — a lot of the output of our work and activity as humans in fact — is in the form of words. So what can we learn when we apply machine learning and natural language processing techniques to text?

      The findings may surprise you. For example, did you know that you can predict whether a Kickstarter project will be funded or not based on textual elements alone … before it’s even published? Other findings are not so surprising; e.g., hopefully we all know by now that a word like “synergy” can sink a job description! But what words DO appeal in tech job descriptions when you’re trying to draw the most qualified, diverse candidates? And speaking of diversity: What’s up with those findings about differences in how men and women describe themselves on their resumes — or are described by others in their performance reviews?

      On this episode of the a16z Podcast, Textio co-founder and CEO Kieran Snyder (who has a PhD in linguistics and formerly led product and design in roles at Microsoft and Amazon) shares her findings, answers to some of these questions, and other insights based on several studies they’ve conducted on language, technology, and document bias.

      Show Notes

      • How analysis of language can predict success on Kickstarter, affect job listings, and more [0:00]
      • Specific words and phrases to use and avoid [9:59]
      • Discussion of how the analysis works [16:11], and how language can affect gender bias [23:59]

      Transcript

      Sonal: Hi, everyone. Welcome to the “a16z podcast.” I’m Sonal, and I’m here today with Michael, and we are talking to Kieran Snyder, who is the CEO and co-founder of Textio, a company that analyzes job listings to predict how well they’re going to perform, and can help optimize them to get more qualified, diverse candidates. And interestingly, they’ve been able to figure out, besides what doesn’t work very well in job descriptions — words like synergize — they’ve been able to figure out what does work well.

      Broad effects of language

      Kieran: Language, like — in tech, people love to talk about hard problems and tough challenges.

      Sonal: But it’s a lot bigger than just about jobs. The ability to understand the words we use and how we use them is pretty important, because even though we’re completely immersed in a world of tech, where a lot of the conversation is around big data as numbers, a lot of the data that we produce — or, the output of our work — is actually taking place in the form of words, and those words matter.

      Kieran: Sometimes how you say things is more influential than what you’re actually saying, right, and it’s counterintuitive to any of us who’ve built products before, because you like to think you’re leading with a strong vision.

      Sonal: Clearly, words matter. And another place that that plays out is with hidden biases that are often revealed in words. For example, Kieran examined a number of resumes to see the differences between how women and men describe themselves, as well as in performance reviews, to see the ways that women and men were described differently.

      Kieran: The word abrasive, which has been talked about since then, ended up, you know, being used in 17 out of a couple hundred women’s reviews, and 0 times in men’s reviews, right. The, sort of, stereotypical, like, “aggressive” was used in a man’s review with an exhortation to be more of it, and in women’s reviews, it’s a term of some judgment.

      Sonal: Okay. Let’s get started. Kieran, welcome. So, the reason we actually invited you to the “a16z Podcast” today is because you’ve been writing a lot of interesting work based on the outcomes of your product, where you’ve been analyzing people’s use of language in certain contexts as a way to surface insights. And I think that’s really fascinating, because I think we have a tendency, in our world, to focus on big data as if it’s just numbers — and not other forms of data, because you’re really describing — I mean, what you describe your work as doing is applying machine learning to text and natural language. So, how did you kind of — how does that work, and then we can talk a little bit more about how you got there?

      Kieran: Yeah. So, how does it work? Language is just an encoding of concepts, right, and anything that can be encoded can be measured. And so, I was sharing this story the other day — we were actually — originally started out looking at Kickstarter projects, right. So, we started out with this question — could we just look at the text of a Kickstarter project, and some of its, you know, metadata around the text and predict, you know, before it was ever published, whether it was going to raise money. And we didn’t look at the quality of the idea. We didn’t look at whether a celebrity endorsed it. It turns out we got over 90% predictive on minute zero of a project, as to whether it was going to hit its fundraising goal, based solely on things like how long is the text, and what kind of fonts are you using, and how many headings do you have.

      Sonal: So, wait a minute, just to unpack that a little bit. So, before the project even went live on Kickstarter, just looking at those features of the text, you’re able to predict whether it [will] be successful or not.

      Kieran: Exactly.

      Sonal: What were some of the high-level takeaways from that?

      Kieran: Yeah. So, longer is better where Kickstarter is concerned.

      Sonal: Interesting.

      Kieran: Kind of counterintuitive. One thing that broke our hearts, because my cofounder, Jensen Harris, and I both have some design background — you would think these cleanly designed projects, with this beautiful use of single typography would do best. Not so. You want it to look like a ransom note. So, you want to mix and match types. You want lots and lots of headings.

      Sonal: Oh, my God. That sounds visually painful.

      Kieran: You want images to be frontloaded, kind of makes sense. But a lot of what we found was not intuitive.

      Sonal: Interesting.

      Kieran: And so, it demonstrated for us the value of actually measuring, because the whole Kickstarter corpus is out there in the world, right. So, you can actually have great training data. You can see how well prior projects have performed. And we saw, “Hey, we’re kind of onto something here,” just looking at the — so very painful as a product person, the quality of your idea doesn’t matter — just looking at the content aspects we could predict.

      Michael: And how do you account, then, for all the other sort of outside variables, you know, whether it was at the beginning of the Kickstarter kind of, like, craze, whether it was a certain time of year for that matter?

      Sonal: A certain type of product even.

      Michael: Yeah. Or geography? How do you know that, in fact, your analysis was correct?

      Kieran: I mean, you can look at some of those other factors, right, because you can see when projects are published. It turns out that doesn’t make a big difference. You can see — the only things that really moved the needle in a very short-term way are, do you have a celebrity endorsing you — because that can get you a lot of social media attention. It doesn’t make or break you, but it can help quite a bit. And generally, how good you are at your social media strategy can tip the balance a bit. But none of those other factors turned out to be as significant as we expected.

      Michael: The ability to really zero in via just the text — did that surprise you?

      Kieran: I mean, we started off with a hypothesis that it would be that way, and that, you know, sometimes how you say things is more influential than what you’re actually saying, right. And it’s counterintuitive to any of us who have built products before, because you like to think you’re leading with a strong vision. We weren’t surprised. We were curious, as we started to apply the technology to some other verticals, whether it would extend. You know, our first big area has really been in the area of job listings, where we’ve looked to see in the first real product application — where we’ve looked at listings now from over 10,000 different companies. 

      We’ve measured who’s applied to which listings, and we do see — the content matters. We do see some tailoring by geography. It turns out what works in New York is different than what works in San Francisco. We see a lot of tailoring by industry. So, what works to hire in tech is very different than what it looks like to hire a claims adjuster, or someone in retail, right. So, you see some differentiation. But in all cases, depending on how you’re slicing and dicing the categories, that text leads — you know, we’ve looked at real estate a little bit prior to launching our jobs application, and we’ve seen the same principles apply.

      Sonal: So, so far, you’ve been talking about the form of the text — like, the length and the fonts and the design — but, like, were there particular words that popped out as well, in terms of what people said on those Kickstarter descriptions, or anything like that? I’m bringing this up, because there’s just this recent anecdote in the news that I read, about someone saying that you can predict success or default of loan applications based on words people use — like God, or using God a lot will actually mean you’re more likely to default on your loan, for example.

      Michael: By God, I’ll pay you every month. I promise. Yeah.

      Kieran: In Kickstarter, we didn’t look at that. We started looking at that for real estate listings and then jobs, where we’ve looked at it quite a bit. So, we saw when we were prototyping out the real estate stuff that if you say “off-street parking,” that really moves the needle for low-income homes. But for high-income homes, in terms of the number of people who go to your open house, and then the eventual sale price of your home — for higher-priced homes, it’s actually a negative, because why would you want to highlight that it has off-street parking? It’s just sort of an expectation. So, we saw, you know, vocabulary mattered quite a bit. In jobs, it matters hugely. You know, we’ve identified, at this point, over 25,000 unique phrases that move the needle on how many people will apply for a job, what demographics, how qualified they are.

      Sonal: Could you share some of that insight with us, because, you know, the reason I came across your work is because I read an article about how you analyzed performance appraisals and job descriptions for insights about what moves the needle, and the differences in how people communicate. What are some of the things — I mean, just because we have a huge audience that does job descriptions.

      Michael: That needs to hire some people.

      Kieran: Yes. That needs to hire. Yeah. So, there is, sort of, a set of language that works really well for everybody. These are not surprising on the face of them, but when you look, you see lots of them. So, things like, “We’d love to hear from you.” Be really encouraging and positive in your listing. Using the right balance of talking to the job seeker. So, your background is in science, and you really enjoy roller skating in your free time. And talking about the company. “So, we stand for this,” in terms of the balance between “you” statements and “we” statements, can matter. You know, language like — in tech, people love to talk about hard problems and tough challenges. Curiously, we see patterns change over time. So, my favorite example of this is the phrase big data. So, a year and a half ago, if you used the phrase big data in a tech job listing, it was positive. You know, it was seen as compelling and cutting-edge. In June of 2015, it’s not negative, but it’s totally neutral.

      Michael: That’s interesting. I wanted to ask, because if everybody, sort of, gloms onto these best practices, how then does the signal versus the noise shift?

      Kieran: Exactly. Marketing content, as with any marketing content, the patterns that work change as they get popular and get adopted. And so, one of the reasons we believe software is so interesting as a solution here, is that it can kind of keep track at broad scale of what’s actually happening right now in the market. So, you may have published a job listing that worked really well a year ago, and probably have a lot of your listeners write their job listings as they go back to that one, and then they try to edit it and tweak it a little bit and fix it.

      Sonal: That’s exactly what happens.

      Kieran: Right. But it actually doesn’t necessarily work, because the market has changed. And so, there’s a lot there.

      Key words and phrases

      Sonal: Were you ever — I mean, I’m just curious about this — were you ever able to find or study associations between people’s intent and outcomes in job listings? So, for example, one of the things that we’ve seen happen a lot is that people only become real about what they actually want out of a job description when they actually put words to paper, and words have that power, to sort of help discipline what you’re looking for. You might not even know what you’re looking for until you write it down. Have you ever looked at anything around that, or found — heard interesting anecdotes around that given your work?

      Kieran: We have seen that listings tend to perform better when they are originally authored. So, you can see some degradation over time when people patch, you know — I take a little bit from this listing and a little bit from this one, and I sort of stitch them together. And it’s probably because when you’re originally authoring it, you bring that coherent point of view.

      Sonal: That’s really interesting.

      Kieran: So, a little bit — pretty early for us to have seen that. And we also identify phrases that torpedo your listing.

      Sonal: Like?

      Kieran: Corporate sort of clichés and jargon.

      Sonal: So buzzwords, basically.

      Kieran: One of the very common — we call it a gateway term — that kind of torpedoes your listing is the word “synergy.”

      Sonal: Oh, my God. That should torpedo any piece of content.

      Michael: Yeah. Yeah.

      Sonal: I don’t care what it is.

      Kieran: But it’s a gateway term, because when people include “synergy,” they’re also significantly more likely to include, you know, “value-add” and “make it pop” — kind of silly, but they’re all over the place. And it turns out, every candidate of every different demographic group hates them. And so, there’s a lot of opportunity to improve in these jobs.

      Michael: So, in the, sort of, the editorial world, we would call that jargon. And it sounds like…

      Kieran: We also call it jargon, specifically.

      Sonal: I think we all call it that. Jargon is jargon. No, totally. Actually, it’s interesting, because, with words like that, they’re obviously in use because they’re useful words, and it’s kind of sad, because — I mean, synergy at some point was probably a useful word. So, it’s kind of interesting, because over time, with your corpus of data, you’ll be able to sort of map how people’s language changes.

      Kieran: Exactly.

      Sonal: And when you think of dictionaries as, like, these static instruments for capturing text these days, it is kind of fascinating how language is changing in a way that we’re able to track differently now, thanks to online and software.

      Kieran: It changes lexicography, like, just as a whole discipline. It changes lexicography for sure. I don’t know that you could do it in a static way anymore.

      Sonal: Right. I totally agree.

      Kieran: The internet has just exploded that.

      Sonal: Right. Exactly.

      Michael: So if big data is, kind of, neutral now, is there a kind of job type or job description that’s the celebrity of the job search world right now?

      Sonal: Yeah. What word is, sort of, popping out that’s really moving the needle for you guys, or that you’ve observed?

      Kieran: There are several. Most of your listeners are probably in tech. It varies a lot by industry. So, “at scale” right now. “At scale” is a very popular phrase.

      Michael: A-ha.

      Sonal: That’s popular here, too. We talk about that a lot.

      Kieran: Yeah. Well, it is. You don’t want to do things and use methods that are perceived to be manual, or perceived to be limited in some way. So, “at scale” is one that shines — and it started in tech, but it spread to other industries, which is common that we see that. One of my favorite examples, given that we spend a lot of time talking to HR people, is — turns out “workforce analytics” is no longer a good phrase to use. You want to use “people analytics.” So, you know, you can get these highly specific, you know, deep in an industry changes — that if you’re in the industry and you’re on the cutting edge, you probably know, but if you’re just a startup trying to hire your first analytics person, you probably have no idea. You don’t have a deep background in the industry.

      Sonal: That’s great.

      Kieran: Right. Yeah.

      Michael: So you’ve described different job listings in real estate. And so, this approach you think can extend in different directions. You started with Kickstarter, but what is it that it’s doing, and how do you — like, it seems a little bit magical, I have to say — that, like — I know that this is a job listing, so therefore, it’s going to have to do this. But a real estate listing has to do something kind of different.

      Kieran: Right. That’s a really good question. So, you know, this approach is as powerful as the data set that you have. So, if you want to understand a document type, the very first thing you need to do is collect a lot of examples of the document type. And that means you need the documents, and you also need some information about their outcomes. So, you are publishing a Kickstarter project. We want to know, did you make money or not? That signal for us. You’re publishing a job listing. We want to know, did you attract a lot of good people? Did you attract only men? Did you attract no one? So, you know, for each document type that we take on, the first thing we do is, we make sure we build out a great training data set. 

      And then we apply really classical natural language processing techniques. So, we look for patterns, and we say, “Okay. These are the ones that were successful,” where successful is defined as, you know — attracted more applicants than 80% of similar listings, maybe. And then we start looking for the linguistic patterns in the successes, the ones that aren’t as successful, ones that skew in a certain way demographically, and then we play that back. So, sort of a key thing for us, is that you get that feedback in real time, as you’re typing. So, as you’re working on your document, before you ever publish it, pay to publish it somewhere, you can make it good. And so, the training set is the, sort of, core of all of that, because without that outcomes data, then it’s just someone’s opinion.

      Michael: And then could you extend that to say, like, “Look, I want to write a screenplay for a blockbuster.” I mean, could you — people have probably tried this, but…

      Kieran: In fact, a very prominent Bay Area CEO proposed to us a couple months ago that we start applying this to screenplays.

      Sonal: To actually start producing content, or just analyzing them?

      Kieran: Sell it to Hollywood.

      Sonal: Oh, wow. That’s great.

      Kieran: Yeah. So I think any time you’re writing content to sell something, this is really interesting technology. And you could be selling your company. You could be selling yourself — you’re a job seeker with a resume that you want to have optimized. You could be selling your product in an e-commerce setup. You could be marketing yourself. You could be marketing blast emails. Any time you’re writing content to get people to take an action, this is really useful technology.

      How the analysis works

      Sonal: Well, let’s talk about where this fits, and let’s purposely use some jargon here, and let’s talk about where it fits in the tech trends — like, where it fits in that space. So, it sounds like you’re describing — big data techniques apply to natural language, or machine learning techniques applied to natural language. But natural language has been around for over 3 decades, 30 years. I mean, in the early days, they didn’t have this kind of corpus to train the algorithms on, obviously, so they had to use different kinds of techniques. Like, where does your work fit, and how do you see how it fits in the evolution of natural language — like, how has it been and where are we now, kind of?

      Kieran: Yeah. I mean, I think in core natural language processing, empirical strategies have always been really important. So, when I was a grad student years ago, writing a dissertation, collecting data was just a lot more work, right. So, I had to go and record people in the field, and I had to transcribe things. I mean, it feels ancient now, actually, but I actually finished my Ph.D 12 years ago. It wasn’t that ancient. The fact that the internet has codified everything over the last 15 or 20 years, at least in English and most Western languages, means that you have this ready set of corpora available for you. The tricky part is collecting the text and the outcomes.

      Sonal: Right. 

      Kieran: The outcomes are the part that’s hard. Finding the content is easy.

      Sonal: So, you’re describing the difference between just analyzing something and being able to predict something using that text.

      Kieran: Exactly. When you analyze something, you can say, “Oh, cool. This word is really popular now. That’s an interesting fact. It might be valuable to someone to know it.” But it’s different than saying, “This word is actually helping your document in some way.”

      Sonal: What are some other scenarios where you could use, sort of, this natural language text analysis to predict interesting things?

      Kieran: Yeah. So, people are really starting to think broadly about this. We saw a New York City-based company helping people optimize the sale of their New York City apartments recently, using the right phrases. We’ve seen people do things in healthcare that I think are really interesting. It’s not a known vertical to me, but looking at the kind of notes that doctors take about a patient, and predicting the patient’s likelihood of having a major insurance incident over the next, you know, 12 to 15 months. Some really interesting things in actuarial science. Like, I think anytime people are producing text — which, by the way, in businesses, whatever your business is, text is actually the thing you produce the most of…

      Sonal: Right. I believe that.

      Kieran: …which any industry, and so people produce a lot of text. It’s meant to describe often what they think is going to happen. And so, I mean, the field of opportunity is pretty big.

      Sonal: The techniques you’re describing — is it the same underlying technique applied to all different domains, but do you have to also train each corpus on a different domain? Like, there’s a special inside language in each industry. Or are there also universals across all of them?

      Kieran: That’s a really good question. You don’t know until you train, is the short answer to the question. So, we have a set of NLP libraries that look for common attributes of text, and we always start out any new vertical by turning them on the documents and seeing what happens. So, things like sentence length — almost always interesting. Things like the density of verbs and adjectives — almost always interesting. Document length — almost always interesting. But the specific phrases that matter, and what it means to write a job listing, is very different than what it means to predict whether a patient is going to become ill, right. 

      And so the specifics matter. The goals matter. So, if it’s a document that’s intended for broad consumption, it really probably shouldn’t be longer than 600-700 words. If it’s a stock prospectus, where you’re giving a company some information about how their stocks are likely to perform, it’s going to be pages and pages. And so, you know, the specific benchmarks that you’re looking for often vary vertical by vertical, but the principles of the kinds of things you look for are pretty similar.

      Sonal: In the past, it seemed like only really big companies could do this, because they had, like, the type of computing hardware and processing power to pull this off. Like, what’s changed that a small startup could do this?

      Kieran: AWS. AWS is what has changed things, right. I mean cloud compute at scale and, you know, Google Cloud and Azure. There’s a lot of competitors now, but AWS did this for startups, I think. And I say that, not because I worked at Amazon before, but it actually is. Like, for our team to set up the server infrastructure that we need is [critical]. You know, so I think that that’s a thing. And just the fact that there’s so much text data encoded on the internet. Google has democratized a lot of access to data. And so, that has helped, too.

      Sonal: That’s great.

      Kieran: Yeah.

      Michael: Did you guys, I have to ask, did you kind of put any Kickstarter projects up there yourselves, just to give it a whirl?

      Kieran: No. We were asked this a lot during our fundraising. We did look at pitch decks, by the way. One of the things…

      Sonal: Oh, I want to hear about that, by the way.

      Kieran: I will come back to your question. One of the things that’s been fascinating about having the beta out there in the world is the ways people are using it. So, of course, they’re using it for job listings, but people are using it for everything. Like, just a couple days ago, I had a material science professor write to me saying, “I put all my course syllabi through.” I was like, “Really? Like how did that work for you? I can’t imagine that that was a good result.” And he’s like, “Oh, I threw out all of the job parts. I just looked at gender bias, because that was a component that I needed for what I was doing.”

      Sonal: Wow.

      Michael: So, describe, when you say put it through — like, what happens? I understand, like — in my head, I have this idea that I’m typing along and, you know, suggestions come flying at me, but…

      Kieran: That’s exactly what happens. So, there’s a website, and you paste or type in your content, and as you’re typing it’s getting annotated and marked up for you with patterns, suggestions, things you might want to change, scores.

      Michael: And you can, in the case of the syllabi, right, you can dial it up or down depending on what you want the outcome to be. So, in his case, “Look, I’m sort of tracking for gender bias or…”

      Kieran: He was looking for a specific aspect of what we provide. And, of course, the product isn’t tuned for what he wants, but he still found that aspect to be applicable to what he was doing. We’re seeing people put marketing content through, pitch deck content through. So, to your question, about did we initiate any Kickstarter campaigns? We didn’t because we weren’t making…

      Michael: But you guys would be genius at it.

      Kieran: We might be, yes. We’ve given a lot of advice to people on Kickstarter projects since then. But we didn’t, because we were making an enterprise product, right, and if we had followed through on a Kickstarter product and then it got funded, then we’d have to build it.

      Michael: Right.

      Kieran: But we helped friends, for sure.

      Sonal: That’s great. So, what did you find out about the pitch decks actually? I’m totally intrigued by that, obviously, given who listens to our podcast.

      Kieran: I mean, pitch decks are not always highly text oriented, right. So, great pitch decks don’t include just your text attributes, but there are certainly things like length of your deck that matter. Slide titles end up mattering quite a bit, because people are looking to see a certain style of content.

      Sonal: And less space. And we’ve all seen any kind of meeting where some one person gets hung up on one word in a headline.

      Kieran: Yeah. It can.

      Sonal: It always happens too.

      Kieran: It can. We didn’t go deep on pitch decks, but we looked at as many as we could find as we were building our own pitch deck in our last round of funding, and found some patterns in the set.

      Michael: In the synergy line of questioning, were there words or phrases you should never include in your pitch deck?

      Kieran: You know, I don’t know.

      Michael: Okay.

      Kieran: I don’t know.

      Sonal: I guess, there might not even actually be — yeah. I wonder if there’s — there’s never, I guess, a set set of rules.

      Kieran: I bet there are. We didn’t identify them.

      Michael: Right. Synergy is probably one.

      Kieran: Yeah.

      Language and gender bias

      Sonal: Actually, let’s talk a little bit more about — and maybe we should wrap up on this note — let’s talk a little bit more about some of your findings around gender differences.

      Kieran: Sure.

      Sonal: So, you said the materials science professor tested his own syllabus — which again, I’m not sure that made sense, like you said, because there wasn’t a reference corpus to, I guess…

      Kieran: There wasn’t, but when you have, you know, tens of thousands of phrases that are lighting up, and he’s writing for a science STEM student population, odds are good that there’s going to be some lexical overlap.

      Sonal: Oh, that’s great. Right.

      Kieran: So, you know, he found some things there.

      Sonal: So, describe some of your findings around job descriptions, because — given what your product focuses on right now in terms of gender differences — and how people — what things you picked up on that?

      Kieran: Yeah. So, prior to us doing this, there was some really strong qualitative research, right. The National Coalition of Women in Technology, the Clayman Institute here at Stanford — they’ve done some really interesting qualitative work, but the number of phrases that they identified was on the order of a couple hundred. Avoid “rockstar.” Avoid “ninja.” You know, we want to hire more women in technology.

      Sonal: Guru.

      Kieran: The interesting thing for us — first of all, we’ve talked to a lot of industries outside of tech. And so, while in technology we want to hire more women, when I talked to people who are hiring ICU nurses, or elementary school teachers, bias goes the other way. And so, it’s very important to us that we don’t judge — we just forecast and let you make the right choices for your business.

      Sonal: Right. Whatever you’re optimizing for given wherever there is an indifference or imbalance.

      Kieran: Right. Right. So, I will say, we have validated much of the qualitative research, which is good, that there’s, you know, some alignment on those points. We have found cases where things are — it’s pretty subtle, right. So, the difference between “fast-paced environment” and “rapidly moving environment” — it’s almost head scratchingly tiny, but statistically, one of them…

      [End of Transcript]

      • Kieran Snyder

      Startups as Science Experiments — Can VC Disrupt Academia, and Vice Versa?

      Marc Andreessen and Vijay Pande

      It’s a myth that startups happen in isolation. Those legendary two people in a garage are often building on the deep and basic research — long funded by government and conducted in universities — that has come before it. But with the advent of the internet, what’s the future of peer-to-peer collaborations and startups-as-“science experiments”? Can venture capital disrupt academia… and vice versa? And finally, what’s the secret to universities like Stanford making money off the entrepreneurial ideas coming out of them? (Hint: It starts with a ‘p’. But not what you think.)

      a16z’s new professor in residence Vijay Pande interviews Marc Andreessen at our 2014 Academic summit on these topics, as well as ‘regulatory arbitrage‘, how to mix humanities and science, and what Marc would have majored in if he were 18 today.

      Show Notes

      • The future of research and the influence of philanthropy [0:00]
      • Merging computer science with other disciplines in education [10:07]
      • How tech varies between West Coast and East Coast, as well as abroad [16:35]
      • The future of AI and ML [19:47]

      Transcript

      The future of research

      Vijay Pande: So, I’m Vijay Pandey, I’m a professor at Stanford, but also in an interesting new role here as a sort of professor-in residence at Andreessen Horowitz. Towards that end, you know, there’s a couple interesting things that we can think about for how venture can disrupt academia, and how academia can be disrupted in an interesting way. And so, one way that we talked about is related to Bill Janeway’s hypothesis that investing, especially long term basic science investing over decades, is really what’s responsible for the success that we see in IT, and that biotech hasn’t had that quite that time yet, and that cleantech has really had nothing close to that. But now, with government probably not going to be able to put the same type of money and emphasis into things, how’re we going to have the seed corn for the future.

      Marc: Yeah. So, this is — for those of you [who] haven’t read Bill, so Bill Janeway is a venture capitalist, and actually a Ph.D in economics who studied from a student of Keynes. And so, he’s kind of straight in the, what is it, the Oxford sort of lineage of economics. So, he’s both a practitioner and academic. And his book is called “Doing Capitalism,” and it’s probably the best single book on the theory of venture capital that I think came out last year. It’s one of his books that came out from academic press, and so it’s got, like, a terribly ugly cover and like no marketing. And so, nobody’s heard of it. But it’s fantastic. It’s an absolutely outstanding book. What he basically observes, he says, look, venture capital has tried to engage in all these categories. IT has been a huge success. It turns out, four decades of federal R&D money preceded venture capital success in IT. Biotech has been a moderate success, two decades of federal R&D money. And then everything else has just been a train wreck, with cleantech being the most recent train wreck. And he says, look, there was no federal research funding, right? The federal government went straight to the industry subsidies without passing through R&D. 

      And so, consequently, there was nothing to draw on. There was not enough science to draw on. And so, he kind of makes the, you know, sort of interesting, profound, potentially disturbing statement of — if you want to think about the future of entrepreneurial capitalism and venture capital and startups, look where the federal R&D money is, and basically invest, you know, behind that between 20 and 40 years. Which also, by the way, goes to something I’ve observed, I completely agree. Somebody earlier said such a William Gibson quote, that the future is here, it’s just not widely distributed yet. I have just [been] continuously struck by the number of times the hot, new innovation that we see — or the hot, new innovation that becomes, you know, this huge thing, is something that was running, virtually, invariably was running in a research lab 20 years earlier. By the way, often 30 or 40 years earlier, right? I mean, in a lot of ways, like, the, you know — Facebook, you know, very successful company, right now. In a lot of ways, Facebook is <inaudible>, right from University of Illinois, right? 50 or 60 years later, right there. You know, these ideas play out over very long periods of time. And a lot of that has to do with the early research. So, the bad news is, right, to the extent that the research funding is not what it should be. And I think it’s imperative on all of us in industry, you know, to try to push for as much basic research funding as possible, because that clearly is the key.

      Vijay: What else can you do?

      Marc: Well, so I think it also — I forget who said it earlier, I think maybe Balaji said it earlier — which is I think the future — the hope, the optimistic view — in lieu of lots of federal research money, the optimistic view would be a more, I would say, open network, collaborative peer to peer approach to research development. You pull on the following threads. You would say, you’ve got the internet as, like, a new — basically coordinating mechanism for lots of smart people all over the world to be able to collaborate and share information and sort of build on each other. You’ve got open source. Literally open source, like open source software, but also the open source mindset — open source data, and you know, open source, you know, all these other things. Designs that could be open sourced. So you’ve got that as a thread to pull on. 

      You’ve got globalization as a thread to pull on. And that there’s just more people worldwide now engaged in research and development than ever before. And collaboration between countries is going to be very powerful. And then you’ve got this really interesting kind of intermingling between research and development. Again, going back to historical, you know — arrows going back to original natural philosophy or going back to the original engineering, which was like, let’s go try to make something — and then let’s derive, you know, principles out of what we’ve tried to make. And so — and then industry — and, you know, for all of the criticisms that, we all love, you know, <inaudible> against either venture capital backed startups or big technology companies — you know, generally speaking, there’s a worldwide boom in industrial companies and technology companies getting built on the basis of R&D. 

      And so, a very — I would say, imperfect — like, you know, since the 1950s style top down, you know, there shalt be the NIH and NSF. And then 30 years later, you know, Microsoft appears. In a sense, like, that’s very idealized, and that’s very predictable, and that’s very wonderful. It may be that we’re just gonna be living in a world where it’s going to be much more bottoms up, much more collaborative, much more diffuse, much less well organized, much sloppier. Maybe, by the way, more dynamic, maybe more creative, maybe more intermingling between disciplines. 

      Vijay: What do you think of this crazy idea. So, let’s say, you know, a VC firm wanted to put in $50 million, in terms of fifty $1 million seed funds, right? And maybe with a relatively low valuation, because it’s very early stage. So, it’s something that you normally would stay away from in science projects, but at that sort of small stage, seed stage, you could actually see. And actually, if you had a big enough valuation, it would be worth [it] for you later on. Do you see yourself doing that? Or what’s the issue with that? Maybe there’s just all the bandwidth to facilitate that. 

      Marc: So, we do some of that. And actually, I would say in the Valley, Khosla Ventures is probably the most advanced on this. They actually call them science experiments. By the way, the terms are like, you know — literally, like, they’ll invest a million dollars at a $1 million pre-money valuation.

      Vijay: One on one?

      Marc: One on one, yeah. So, you know, you have to like…

      Vijay: That makes one on two pretty good.

      Marc: One on two looks great. Yeah, exactly. We might even offer one on three. So, there’s a little bit of that. I guess the counter argument on that would be that there is a difference between science and technology, there is a difference between research and development. It’s not clear that research as research benefits from having any, you know, short term commercial. In fact, research is probably compromised by having short term financial incentives. The other lever that I think you can pull — you know, if the question, ultimately, is how to get the $50 million — if the question is how to get $50 million, you know, sort of from the venture capital <inaudible> into research — the lever that you probably pull instead I think is philanthropy. And I think this is the other part of the system that’s working better now than before. And I think there is going to be — I think this will be maybe the big upside surprise in the next 20 or 30 years — the bow wave of philanthropy coming out of the high tech community and coming out of high tech founders and CEOs and people who have been successful in these companies going straight back into universities. 

      I mean, right now, I mean, it’s staggering. Like, you just walk through the Stanford campus, and you just, you know, the Yang center, and you see all the buildings, and it’s just mind boggling. Actually, this week or next week is the unveiling of Ram Shriram, who was an angel investor in Google — just funded a new biomedical research institute. So, there already is, like — you know, Tom Siebel, at University of Illinois has played this huge role. My father in law, actually, at Stanford has played this huge role. And so, there is a lot — I think you could see, you know, 10 times, 100 times the amount of philanthropy from industry and from successes in industry flowing back into universities. And I suspect that might be the real lever. By the way, that goes to something somebody mentioned this morning, which is — the really, in my view, the enlightened universities that really think strategically about things like spin offs, and students and professors taking leave and so forth, are the ones who realized that the long game here is probably philanthropy — as contrasted to, you know, university venture capital, or as contrasted to, you know, patent licensing or whatever it is.

      Vijay: Well, I think Stanford certainly sees it that way. And from what I’ve seen, especially since, you know, this ability to go back and forth is something that is not looked down upon, but it’s something that is actually a real flexibility. One of the questions I have is when thinking about academia, there’s this large spectrum of people we could talk about. We could talk about the undergrads, the grad students, junior faculty, and senior faculty. And while senior faculty are probably the most interesting to talk about , you know, actually, the undergrads are very interesting to talk about. I mean, could you imagine, let’s say — put yourself as a first year undergrad, like, what would you want to be doing?

      Marc: Oh, so I think about this a fair amount. We try to think about this to kind of think about what we would be investing in. So, I would definitely be computer science. For the last 10 years, I thought it would be some sort of combination of computer science and biomedicine. I don’t know enough about biology to really know exactly where I would go, but I would look for wherever the heat is at the intersection of biology.

      Vijay: Why biology?

      Marc: I just think there’s so much — I think, software and big data is going to be such an enormous lever to be used on affecting human health in the future. And we’re making a whole series of investments against that. But I just think that we’re at the very, very beginning of the intersection of those two fields, and the outcome is, you know, the outcomes in people’s lives, whether it’s personalized medicine, or new kinds of medical devices, or, you know, sort of different forms of human augmentation are just going to be really breathtaking. That’d be one. The other one more recently, though, is I think if I were 18 — I think I would be very tempted to go headlong into cryptocurrency and into distributed systems. I think that there’s the potential. I think if we were to create the internet today, I think we would do it completely different. I think it would have it be completely decentralized. And I think we would have cryptocurrency built in at the core, and I think it would be far more robust, and would have all kinds of interesting properties that it actually doesn’t have today.

      Vijay: So, no more 404s.

      Marc: Yeah. But like, you know, you build in the concept of monetization. So then you can do — you know, all the issues with resource allocation on the internet, starting with spam and going all the way through to things like quality of service. Basically, computer science meets economics would be a huge opportunity.

      Vijay: So, why not just start that now?

      Marc: Well, you know, the internet does kind of have the snowball rolling down the hill kind of thing going for it, for all of its issues. I mean, and the answer is, by the way, we are, to the best of our ability, we are — and there are a bunch of startups trying to recreate DNS. There are startups trying to recreate, you know, file storage. And I mean, even like, you know, there’s applications even for things like, sort of, countering censorship regimes. If you could do peer to peer routing, then all of a sudden the Great Firewall of China doesn’t matter so much. And you see more and more of these ideas popping up in startups now.

      Computer science vs. other disciplines

      Vijay: Yeah, that sounds fantastic. You know, it is exciting to see, with undergrads being so excited about CS and [it] becoming a dominant field. I think one of the things that’s appealing, I think, is I see what entrepreneurial power you can have with a CS degree. But I was always very curious to see, too, how that could be broadened. So, you talked about biology, and I can imagine science has been entrepreneurial. If you had to advise people for how to have entrepreneurial liberal arts — let me tell you my motivation here, which is that, you know, I think all of us have colleagues that look at computer science and see all the great things it’s doing. And it’s both exciting, and maybe a little scary, and computer science is dominating in terms of undergrads, and so on. And I don’t think any of us want to see that other part of the university disappear. That’s something we’d love to help. 

      And I think the entrepreneurial aspect of computer science has been very powerful. So, to give examples, I mean, so, in social sciences — you can imagine computational social sciences becoming big, especially with big data. I’ve been trying to think. So, this is something where maybe we’re not going to solve the problem over 30 seconds, but I try to think about how we can sort of incorporate other aspects. And there’s creative aspects, in terms of art and dance and things like that I can imagine being part of it. But if we can find a way to, sort of, take that entrepreneurial spirit and apply it to other parts of the university, I think that would be disruptive in itself.

      Marc: Well, I think for sure we can bring computer science and science more broadly, and math, into more liberal arts fields. But the arts alone, you know — music and movies, sort of, having a huge impact, and there’s people doing very interesting work. I will tell you, I’m an unabashed bigot — and I probably have a lot of people who won’t argue with me in this room, although this is a really good way to clear out a dinner party, if you haven’t tried it, in about 10 minutes. You know, there’s two basic, I think, aspirational modes for universities. One is, sort of, the classic life of the mind, you know, kind of Allan Bloom, you know, philosophy, science, or philosophy and liberal arts. The other, I think, you know, engineering, learn how to make things. Maybe a Midwestern farm boy — maybe we were just raised this way — but I think making things is what you do as your job, and the life of the mind is what you do for fun.

      There’s this book that is coming out of this, is it a Yale professor? Dershowitz is writing this book. And basically this book, it’s gotten some heat on it. He’s like the ultra left wing alternative to Peter Thiel, will be the way to think about it. Basically, it’s a comprehensive condemnation of basically what he views as the usurping of the academy for anything other than the life of the mind. And it’s this, like, searing indictment of today’s undergraduates for, like, being too focused on, like, professional success and being too focused on gaining practical skills and being too focused on learning how to make things. And I just think he’s out of his mind. Like, I just think he’s, like, completely crazy. It feels like this is really going to intensify as an issue unless we figure out how to bring technology and science into more of these fields.

      Vijay: I think that’s exactly right. And I think this is a tension that we see right now in the academy, and I think the solution is not to have one side dominate, but to sort of figure out how we can raise everything. One other issue that comes up that’s very important is the tension between technology and regulation. So, with me spending a lot of time with drug design, you know, if we had to do clinical trials for Google the way we do for drugs, they’ll never be able to make any sort of changes or improvements to their search engine. It’d just take too long. You can’t do AB testing the same way. And now we’re seeing this and other things in terms of Uber and other areas, they’re trying to push the envelope. I mean, how do you see this tension between regulation and innovation, sort of, panning out?

      Marc: So, I think you — have you talked about Eroom’s law?

      Vijay: Yeah. Oh, no, I didn’t bring it up. I think Balaji did.

      Marc: Okay. Eroom’s law says every four years, the cost to develop a drug doubles and the timeframe elongates.

      Vijay: Absolutely. Eroom is Moore’s law backwards.

      Marc: Yeah, it’s the side of it you don’t want to be on. So, this goes actually straight back to the previous topic, which is, I think societally, we decide collectively, we decide how much innovation we want, we decide how much risk we want, and then based on that, we decide how much innovation we get. And I think that there is a general principle that as societies sort of, you know, advance/get older/mature, you know, we decide we want less and less risk. As a consequence, we get less and less innovation. And then, at some point, it’s sort of like the cycle of civilizations. At some point, the barbarians show up, and the barbarians, like, have less to lose, right? And so they’re like, oh, well, we’ll just go for it. 

      So, it may be that the right thing to do — I mean, there’s kind of two ways, I think, to think about it. One is how to fight inside a system like in the United States, and try to figure out how to navigate through this. And you know, different companies have different tactics on this. The other is to let, you know, basically, go to new regulatory regimes. You go after basically what we call regulatory arbitrage. So, if you can’t do it in the US, find a country where you can do it. And by the way, it could be us finding a country that can do it, or it could just be an industry forming in a country where you can do it that has nothing to do with us, because it just happens to be legal and supported there. And I think stem cell research in Korea has been an interesting example. In drones, we’re seeing this now. You know, it’s still illegal to fly drones in the US, and so the initial drone deployments are all outside the US. So, we’ve been talking publicly about a couple of ideas on this. One is, you know, if you think about a lot of countries, and in fact, a lot of cities in the US a lot of countries around the world want to have their own quote, Silicon Valley, the answer probably is not to have their own Silicon Valley — it’s probably to have a different kind of Valley — you know, biomedical valley or a stem cell valley or, you know, take your pick.

      Vijay: With a friendly regulatory environment.

      Marc: Exactly. Specifically, and basically make a specific decision in a specific area of research to legalize a certain kind of risk, in order to get that kind of innovation in that place. The other — Balaji has been talking a lot about this — would be the idea of taking the old Economic Zone idea, that was used so successfully in places like Hong Kong, and apply that into this domain. And Larry Page has been talking about this as well, which is — maybe we don’t want, like, self-driving cars everywhere all of a sudden, but maybe, you know, a city that wants to have an economic boom around it would decide to legalize it.

      Vijay: Well, and that will help people with the fear that we talked about, that you can sort of ease into things without having something be national law. And the nice thing about the way the US is set up is that there is this possibility for local versus more federal government. And as long as that was possible, I think that could be a way that things could innovate.

      Marc: The problem is, a very large number of people are very invested in the current power structures, and so, the minute you start talking about this, it immediately gets painted as like you’re advocating secession. And it’s like, no. We just want to have, like, a place where we can do something new. It doesn’t need to, like, you know, violate all the rules. It doesn’t need to secede from the US. It’s just like, how about we can just, like, have a drone fly. But back to the C.P. Snow thing, like, the reaction is — it gets very visceral very fast, and that is something that requires a fair amount of energy to fight.

      Cultural differences in tech

      Vijay: There’s also, sort of, an East Coast, West Coast, rap war of sorts. In terms of East Coast, you know — paper belts, newspaper, Wall Street — versus West Coast, you know, computers, innovation. How do you see that changing? Or do you think that those styles are pretty ingrained?

      Marc: Well, to start with, it’s like in hip-hop, it’s descriptive rather than prescriptive, which is, there are plenty of people on the East Coast who are very innovative, and there are plenty of people as opposed to who are very hidebound. I mean, just drive through San Francisco and look at the zoning, and you’ll discover plenty of conservatives who don’t even realize they’re conservative. But with the constraint that it is descriptive rather than prescriptive, there is definitely to that. And I would argue this actually goes deep in the American culture. It has for a very long time. Which is, this is the whole go west, you know, kind of phenomenon, which is, you know, where’s the frontier. It’s not a surprise that we happen to be right here, like — we all, literally, are the spiritual descendants of people who came as far west as they could, literally until they would drown if they took another step. We are the extreme case. 

      And so, what I find so interesting about that, from a cultural standpoint, is — that’s a 150 year old or 200 year old — it’s the urge to go to the frontier. And I really feel like the Valley, you know, really benefits from that to this day. The number of people who pick up stakes from, you know — I grew up in the Midwest, or growing up on the East Coast, or in other countries, and coming here is a wonderful thing. At the same time, I do very much believe it’s a state of mind, not a geographic place. And I think one of the really exciting things about the last, especially the last 20 years — like, way more so today than I think when I came out here — the ideas and the attitude of innovation and new ideas seem to be spreading worldwide at an accelerating pace.

      Vijay: You think so? I mean, what’s an example of that?

      Marc: Oh, just the rise of — I’ll just give you an example. A good friend of mine, Chris Schroeder, wrote another great book I’d recommend, called “Startup Rising.” So, he’s a former internet CEO and State Department official who travelled around the Middle East for a year and a half and has friends in all these countries — you know, Jordan and Egypt and all throughout Syria, and all throughout the — it turns out there’s an explosion of internet entrepreneurship happening all throughout the Middle East. It’s basically unknown. And his book goes through the whole thing. He just got one of the 20 business visas a year to go to Iran. He just went to Tehran and spent three weeks, and it turns out, there’s this amazing, effectively underground internet startup ecosystem in Tehran. 

      And he said the founders are exactly the same as the kind of people who are sitting at the Coupa Cafe in downtown Palo Alto. Interestingly, he said at least 40% and maybe 50% women. He said the gender balance is actually much more gender balanced. And he said they are the most fired up young people you’ve ever met, and they have an unbelievable sense of future possibility. Coupled, of course, with enormous frustration of being, you know, in a country with all these, you know, with all the trade barriers and all the political issues. But the amount of energy, I don’t know. If you’d gone to Tehran 20 years ago, 30 years ago, I don’t know if you would have found a lot of PC startups. It feels like something new is happening. At least he will tell you — and his book is about this — he will tell you this is a global movement. There’s nothing about this anymore that’s confined to Silicon Valley.

      AI and machine learning

      Vijay: Yeah, no, that’s fantastic. Especially [because] the cost of the computer is pretty low, and the availability of the internet means they can hook into anything. We talked a lot about machine learning earlier, and that’s something that I’ve gotten very excited about. I think many people in the room have gotten very excited about. I’m curious to see what you think that’s going. I think often, what people think about machine learning — and this came up in the previous talk — that machine learning is, you know, the dream that we’ll have Hal from “2001,” maybe a little less nasty. But, you know, for me, actually — it came up earlier that people said, “Oh, it’s just doing classification.” I mean, classification is actually fantastic. If I can say, this is the drug I want to take, or this is the stock I want to invest [in], you know, any of those things are really exciting. And so, I see, sort of, machine learning being just classification being exciting. But I’m curious to see how far you see it going. Do we have to get to Hal to be interesting? What’s in between classification and Hal?

      Marc: So, I would say — so, I was at computer science — at University of Illinois ’89 to ’93, after AI had been thoroughly discredited. So, I managed to get through all four years without an AI class. In fact, I think there was still one, but it was definitely an elective, and I don’t think you were encouraged to take it.

      Vijay: So you’ve taken more liberal arts classes than AI classes?

      Marc: I suspect that would not be the case if I were to go through the program today, which that was definitely the case then. So, I’m the wrong person to speculate on where the core science goes. I will say, just kind of to your point, just given the techniques that we have today, and then given the consequences of Moore’s law, and all the other changes happening at the component level, the opportunities to us seem very large to take machine learning — sort of, machine learning meets economics equals Bitcoin equals, you know, financial innovation. Machine learning meets quantified self equals biomedical innovation. Machine learning meets Uber equals — and Lyft and these things — and sensors in cars, means revolutionizing the transportation system, means — which means, you know, potentially a significant answer to the emission problem. 

      And so, we see a whole series of fields in which you can basically now bring machine learning to bear. And in particular, anything where you’ve got the deployment of sensors, so machine learning meets the fact now that you have, you know, 3 billion, you know, going to 6 billion smartphones on the planet, all with cameras. What can you do with that? There’s this breakthrough new iPhone app now, where it literally — you just point the camera at a printed page and it, like, reads you the page. It was just like, you know, if you don’t have full sight, you know, it’s a revolutionary, profound thing that was just not possible.

      Vijay: And maybe not that far away, it doesn’t read it to you, it summarizes it for you.

      Marc: Yeah, exactly. Right, exactly. And so, you take the just massive deployment of sensors, you take the massive rise in big data, you take the ability we have to build these technologies now in a way where they intersect into the real world into people’s lives — financial services, healthcare, education, real estate, transportation, government, law — and then you bring in machine learning on top of that, the sort of cumulative effect of bringing those factors together — those are new things. Those combinations are new. And we have the opportunity to build both products and companies that have never been even imagined before. Even without more significant fundamental advances in machine learning.

      Vijay: Yeah, absolutely. Well, I think we’re out of time. So, let’s thank Marc one more time.

      Marc: Good, great.

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.