Welcome to 16 Minutes, our podcast where we discuss tech trends in the news and their impact on the long arc of innovation. Today’s topic is crypto regulation, and specifically, two recent federal government hearings in the news that were focused on crypto and therefore the related trend of web3. In contrast to the model of web2 — typified by very broadly used but also very centralized platforms run by corporations — web3 refers to the idea of a new internet enabled by crypto that is owned by builders and users.

The first hearing that took place was at the House Committee on Financial Services, featuring six crypto company CEOs and resulting in a five-hour session that prompted headlines like “Congress Gets a Crash Course on Cryptocurrency.”

Then, the U.S. Senate’s Banking, Housing, and Urban Affairs Committee held its own hearing, this time focused on stablecoins, which are privately issued cryptocurrencies that are pegged to a stable asset such as the U.S. dollar, and are used in decentralized financial services.

We’ve covered crypto regulatory issues on 16 Minutes before with a16z experts, including an episode with former federal prosecutor  Katie Haun and former New York Stock Exchange regulatory chief Anthony Albanese. That discussion, which you can find in this feed under episode #50, was about a proposal by the Treasury Department’s financial crimes enforcement arm that included provisions for digital asset reporting.

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.)

All of these hearings are also connected to the broader question of innovation, and keeping the U.S. competitive on a global stage.

So with that context, our guest today is a16z global head of policy Tomicah Tillemann, who before joining a16z served as senior advisor to two secretaries of state. He reports on the hearings and their significance, and gives a quick pulse-check on where we are with crypto regulation right now.

• 

  Tomicah Tillemann is global head of policy at a16z. Previously he served as a senior advisor to now-President Joseph Biden and two secretaries of state.

  Follow Twitter
• 

  Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

  Follow Twitter

Welcome to 16 Minutes, our show on the a16z podcast network where we talk about tech trends that are dominating news headlines, industry buzz, and where we are on the long arc of innovation. 

Today’s episode actually features a look back at the GameStop saga — the stock market drama that some headlines described as a “David-and-Goliath battle” that “upended Wall Street.” 

For quick basic context, here’s what happened: A group of Reddit users mass-purchased and drove up prices of stock in the video game retailer GameStop, forcing short sellers including hedge funds and institutional investors to back out in a short squeeze, pushing prices even higher. But beyond the news, this also portended other, broader trends including redefining the power of retail investors, the phenomenon of meme stocks, and more.  

So in this episode — which is from a conversation that originally took place live on Clubhouse  (and which, by the way, can also be found on the a16z Live feed) —  a16z co-founder Marc Andreessen talks to Ken Griffin, founder and CEO of the hedge fund Citadel, which was a key player in GameStop as both a market maker and investor. You’ll also hear a16z general partner and fintech expert Alex Rampell join later in the conversation.

Griffin also just purchased (in a Sotheby’s auction a little over two weeks ago) — one of the original copies of the U.S. Constitution, an auction in which a decentralized autonomous organization called ConstitutionDAO also bid on buying it; Marc and Ken discuss this briefly at the very end.

• 

  Ken Griffin
• 

  Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

  Follow Twitter

Welcome to 16 Minutes, our show where we talk about tech trends in the news! We have two segments today:

1) The announcement recently that Valve Software, which operates the massive gaming platform Steam, added a rule barring games that use blockchain technologies or that allow users to exchange cryptocurrencies or NFTs – this rule appeared on its “What you shouldn’t publish on Steam” onboarding list for developers. We go beyond the players to the trends at play here, putting the news in context — as is the premise of this show — because it not only immediately impacts gaming developers and gamers using the platform, but has implications for gaming business models and the arc of innovation in gaming as part of the web3 movement.

Our expert guests are a16z partner Jonathan Lai and a16z partner Eddy Lazzarin.

2) The FDA’s announcement last month that it authorized marketing of the “first artificial intelligence (AI)-based software designed to identify an area of interest on the prostate biopsy image with the highest likelihood of harboring cancer so it can be reviewed further by the pathologist if the area of concern has not been identified on initial review.” The FDA reviewed the technology from Paige Prostate through its De Novo regulatory pathway.

We have three expert guests: Eli Van Allen, associate professor of Medicine at Harvard Medical School and chief of the Division of Population Sciences at the Dana-Farber Cancer Institute; a16z bio general partner Vineeta Agarwala; and a16z bio partner Jay Rughani.

• 

  Jonathan Lai is a partner at a16z where he focuses on games, social, and creator economy investments. Prior to joining the firm, Jon led North America investments at Tencent and was a PM at Riot Games.

  Follow Twitter
• 

  Eddy Lazzarin is head of engineering at a16z crypto. Prior to that Eddy was a software engineer at Netflix working on data ingestion systems, and data engineer at Facebook working on growth analytics for Messenger.

  Follow Twitter
• 

  Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

  Follow Twitter
• 

  Jay Rughani is a partner at Andreessen Horowitz investing in bio and healthcare technology companies. Prior to joining a16z bio, he worked for Flatiron Health, which was acquired by the Roche Group in 2018.

  Follow Twitter
• 

  Eliezer Van Allen
• 

  Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

  Follow Twitter

Playing out against the backdrop of a global pandemic (including recent massive surges in regions around the world) is the news that came out a week ago that a candidate “malaria vaccine becomes first to achieve WHO-specified 75% efficacy goal”. While the findings are still in preprint with The Lancet, the resulting buzz and phrases quoted included everything from “unprecedented”, “groundbreaking work”, and “very exciting” to “high expectations”, “highly effective”, and “a hugely significant extra weapon”… A “weapon” in the war against malaria that is — a disease that is estimated to cause over 400,000 deaths each year globally, and predominantly in children under the age of five.

So in this special 2x explainer episode of 16 Minutes (also running on the a16z Podcast), we —Rajeev Venkayya of Takeda Pharmaceuticals, a16z bio general partner Jorge Conde, and Sonal Chokshi — dig into what’s hype/ what’s real about this news, beyond the headlines and beyond the buzz. What does the data tell us, what does the current study phase mean, and what’s left to get to widespread, real-world use? How does this candidate vaccine (R21 from Jenner Institute/ Oxford University) compare to the other malaria vaccine (RTS,S from GlaxoSmithKline)? How do, and don’t, advances in and around COVID vaccines play here? And why has it been so hard to develop vaccines for this particular disease?

Because we also cover (as is the premise of the show) where we are on the long arc of innovation… and this is an innovation story that’s been nearly a century in the making.

Show Notes

• The urgency of finding a vaccine to combat malaria [2:45] and why malaria is so difficult to treat [5:12]
• Discussion of the R21 vaccine candidate and its pending study [10:48]
• Previous vaccine attempts, details around clinical trials [17:48], and the pros and cons of different vaccine approaches [24:27]
• How vaccine effectiveness is measured [28:54] and possibilities for the future [33:35]

Transcript

Sonal: Hi, everyone. Welcome to the a16z Podcast network. I’m Sonal, and this is a special 2x episode of 16 Minutes, which we’re also running on the main a16z Podcast as part of our ongoing such coverage — and it’s all about the recent news, science, technology, problem, and innovations behind malaria vaccines.

While the discussion includes contrasts and comparisons to COVID vaccines briefly – and also plays out against a broader backdrop of the massive recent surge of cases in India and South Asia, as well as new waves in Brazil, Turkey, France, Argentina and elsewhere — the big news that also came out this past week is that a new candidate malaria vaccine is the first to achieve the World Health Organization’s goal of 75% efficacy, according to the announcement from Oxford University

Just to give a sense of how big and buzzy this news has been, some of the keywords that have been quoted by experts in many of the releases and articles have included phrases from “unprecedented,” “groundbreaking work,” “very exciting,” to “high expectations,” “highly effective,” and “a hugely significant extra weapon.”

So, as is the premise of 16 Minutes (which, if you’re not already subscribed to, be sure to find, and follow, in your podcast app), we dig beyond the headlines for what’s hype/what’s real, as well as where we are on the long arc of innovation.

Our expert guests for this episode are Rajeev Venkayya, President of the Global Vaccine Business Unit at Takeda Pharmaceuticals, where he leads full-stack development of vaccines for tropical diseases like dengue, norovirus, and Zika. He’s also been trained as a medical doctor and served as Director of Vaccine Delivery at the Gates Foundation, and was previously at the White House in biodefense.

Rajeev has also shared how vaccine development works in general, including outlining the phases and what was accelerated for COVID vaccines in an episode the three of us did last year, with a16z bio general partner, Jorge Conde, who also joins this episode, and has, in fact, been on all of our vaccine episodes. I don’t know if you actually knew that Jorge, that you’ve been on every single vaccine episode. <Jorge: You keep inviting me back!> Yeah, well, sadly — and importantly — we’ve had to cover many different aspects of this topic over the past year and a half.

And, we’ve covered everything from vaccine nationalism and vaccine hesitancy to vaccine manufacturing and scaling and all about mRNA vaccines and much more. Listeners, you can find all of that at a16z.com/vaccines.

But for this malaria vaccine, I’d actually love to start by hearing from both of you what your reactions were to the recent news. And I’ll summarize the specifics of the news in a moment, but, would love to just quickly hear the big picture for why this matters, from your vantage point. 

The fight against malaria

Rajeev: I think this is a really big deal, because malaria is one of the big three (as we call it) — HIV, tuberculosis, and malaria cause an extraordinary amount of suffering and deaths every year.

All three of them have proven to be very, very tough targets from the standpoint of vaccine development. There’s no vaccine yet for HIV, despite extraordinary global efforts for decades. We have a vaccine that leaves a lot to be desired in the BCG, that we give at birth for tuberculosis. And there is a malaria vaccine that was brought to the world a few years ago from GSK called RTSS — but, that vaccine, while efficacious, has some room for improvement, and this vaccine potentially could be that improvement we’re looking for.

Jorge: In addition to what Rajeev said, I would say two of my biggest reactions was number one, the fact that they seem to have early indications of a vaccine that’s highly efficacious against a parasite is no small feat in and of itself (a parasite is a tricky bug).

And the second one I would just point out is, you know, obviously, vaccines have gotten an incredible amount of attention over the last year or so, given the COVID pandemic. But this vaccine, first of all, has been decades in the making… <Sonal: Nearly a century, I heard!> …nearly a century of effort to try to find a vaccine against this parasite, and this breakthrough didn’t come from the same technology that gave us the COVID vaccine breakthroughs. And to me, what’s fantastic to see is, the dividends that come from decades of research that pay off using, sort of, “old-world” technology. What I mean is, you know, this was a vaccine developed using more traditional vaccine production methods, versus what we saw in the course of one short year with the mRNA vaccines that BioNTech and Pfizer and Moderna brought us for the COVID pandemic.

Rajeev: Yeah I think for a long time to come, we’re gonna think about vaccines in terms of “pre-mRNA” and “post-mRNA.”

This is technology that was developed well before mRNA vaccines came of age. The technology that was used is actually proven — it’s been used in the hepatitis B vaccine, as well as the human papillomavirus… <Sonal: HPV, yeah> …yeah, these virus-like particles that are very, very effective at “presenting antigen” to the immune system so that it can recognize it, and then develop an antibody and cell-mediated immune response to that antigen.

The challenge of finding a vaccine

Sonal: Well, let’s dig into what this vaccine is and how they work. But first, just to put this whole problem in scope, just quick statistics. Over 229 million cases of clinical malaria were reported the year before [2019], and the World Health Organization estimated that malaria causes over 400,000 deaths each year, globally. I mean this seems to be concentrated in Africa and South Asia, Southeast Asia, different regions, but it is important to just note the scope of the disease.

Jorge: A lot of these tropical diseases also happen to be “poor country diseases.” I have to wonder if this tropical disease was endemic in the richer parts of the world — had we, you know, gotten to this breakthrough sooner.

Sonal: I was wondering the exact same thing, Jorge. The fact that we got to a COVID mRNA vaccine in less than a year, versus this disease that people have been working on for nearly a century — it almost makes you wonder, like, if this were prevalent in the United States, we would have solved this like 20 years ago.

Rajeev: Possibly, possibly. There would have been a lot more R&D for sure.

Sonal: I frankly think this is also true for women’s health, but I will go on that rant later.

Jorge: I was about to say that.

Rajeev: Yeah. It’s important to note that almost all the deaths are happening in sub-Saharan Africa, and two-thirds of those deaths are in children under the age of five. This is a very, very significant global burden of disease that desperately needs a safe and effective vaccine.

Jorge: What some people don’t realize is, not only is the disease burden high, but malaria is something you can get infected with again and again and again. You don’t get sick once and then you’re immune. There are reinfections. And so that just adds to the burden.

Sonal: I’m really glad you’re bringing up the point that it’s not a one-and-done disease, like another type of disease. I actually read a statistic that the average in a lifetime is six times, that a person can get malaria six times in their lifetime.

Jorge: At some point, you do develop immunity. Like, the reason why people catch malaria an average of six times and not an average of the number of years that they’ve lived — because eventually you become more resistant to it.

Sonal: Right. It’s like my parents, they never get stung or sick. But when we were kids, my brothers and I used to count and compare our mosquito bites, and we’d have, like, competitions for, like, how many we had. I would have (I’m not exaggerating), hundreds of bites on my body. And we did everything by the way. Nets, the coils, everything, you name it. The anti-malarial drugs — although I didn’t take mine, which is why I got sick. I was a young kid. I spit mine out. It was disgusting, and no one watched to see if I swallowed it.

Jorge: And by the way, the way that these anti-malarial pills work (at least some of them), is they essentially poison your blood so that the parasite…

Sonal: They taste poisonous. They taste like poison.

Jorge: Yeah, the parasite can’t survive in your bloodstream.

Sonal: So, why has this disease been so difficult? And in general, is there a difference when you’re designing a vaccine to target a parasite versus a bacteria or other viruses? The reason I’m asking is because, Jorge, you mentioned the malaria parasite — it has a complex life cycle, and can mutate. <Jorge: It’s wild> It’s crazy. I mean I’ve experienced a very mild form of it when I was a child, and just a little anecdotal bit of detail (that I don’t know if people who have never experienced this would ever know this), but in my experience, I had like the highest fever one day; the next day, it was as if I were a completely healthy person. And then the third day, I had a super high fever again. I don’t know if that’s a normal thing, but that was bizarre to me. Like, I had no idea why that even happened.

Jorge: Well, part of the reason why that probably happened is this sort of funky lifecycle of the parasite. So, you know, an individual gets infected when they are bitten by a female mosquito that’s carrying an infectious form of the parasite. That goes to your liver, where it continues to reproduce. Then it gets released to your bloodstream, where it attacks your red blood cells. And, it replicates within your red blood cells, and, when your red blood cells get too full of parasite, they burst, release a bunch of parasites into your bloodstream — that’s what causes the fever to spike. And then, it gets tamped down, and then when there’s another burst (of another set of red blood cells), the fever spikes again.

And then, to complete the life cycle, you now have a sort of premature version of the parasite floating around in your bloodstream, and you get bitten by a mosquito — and now that goes back up into the mosquito (to complete its sexual maturation). So it actually comes full circle. A lot of people think about parasites having a host. You know, in the case of malaria, malaria is being raised in sort of like shared custody between man and mosquito.

Sonal: That’s an incredible explanation.

Do you have any thoughts specifically on what it means to target malaria, as a disease?

Rajeev: Well, it’s almost a self-fulfilling situation, because the fact that a person can have malaria multiple times tells us that the immune system is having a tough time with this parasite. The immune system is not able to identify the parts of the parasite that it can then attack when it gets reinfected to prevent the illness from recurring.

And so, if the immune system, which is super sophisticated, is not able to do that, then almost by definition, it’s going to be a tough vaccine problem. One of the reasons is, the parasite can be quite effective at evading the immune system in the way that it grows in a person’s body. The parasite’s life cycle involves transmission through a mosquito that bites a person who gets infected — but then once that parasite is in a person, it has multiple stages of its growth that can be difficult to target. And so, that’s another unique feature of malaria.

A promising vaccine candidate

Sonal: So, let’s talk specifically about this vaccine. To summarize, the candidate vaccine here, it’s called R21. It was developed by scientists at the Jenner Institute at Oxford. And the reported findings are that they demonstrate a high-level efficacy of 77% over 12 months of follow-up, in a study with African children. Specifically, 450 participants aged 5-17 months, all from the country of Burkina Faso. Most of the doses were administered before the peak malaria season there (three vaccinations were administered at four-week intervals), and then a fourth dose came one year later.

So, that’s a super high-level summary. One more quick note, this was all part of the phase 2b trial — randomized, controlled, double-blind — the findings are still in press with the medical journal The Lancet (and I will include all links and sources that are mentioned in the show notes as always).

Rajeev: One thing I do want listeners to know is that the data we’re discussing today and you’ve seen in the media comes out of a preprint — which means this is not yet a peer-reviewed publication. Other experts will look at the study design and the results and they’ll ask critical questions of the researchers that they’ll then have to address in their responses, ultimately resulting in a peer-reviewed publication. Now of course we do hope that the essence of the findings will remain unchanged, and that the conclusions will largely be the same — but we can’t say that for sure until the peer review process is actually concluded.

Sonal: I mean, the preprint — lately “science-by-press release” is definitely a thing that’s accelerated in the last year for sure. I’ve seen it at a whole new scale that I’ve never seen before.

Rajeev: Yeah, and you know some would say that given that we’re living in COVID times, where literally every day matters in science, that we have to accept science-by-press release, hopefully followed very quickly by peer-reviewed publications. But the peer review and publishing process just takes too long, frankly, for COVID. And so we often end up having to rely on press releases and preprints.

But, I do want to point out that the preprint came out right around World Malaria Day, which is April 25th. And I think that’s probably why these guys released the preprint when they did.

Jorge: Wait, it’s April 25th you said? <Rajeev: Yeah> So World Malaria Day is the same day as DNA Day. That’s interesting.

Sonal: Oh, that is interesting!

Rajeev: I didn’t realize we observed DNA Day. That’s…

Jorge: Yeah, because April 25th was when the Nature article [was published] that Rosalind Franklin published her Photo 51, and Watson and Crick published their one-page, structure-of-DNA paper. And then, in an act of symmetry, the tie between Francis Collins and Craig Venter in sequencing the human genome was announced by Bill Clinton on April 25th.

Rajeev: Oh my gosh, I didn’t realize we were — there was a World DNA Day. Thanks for sharing that.

Sonal: So, back to the point about malaria. Anything more to say on the specifics of malaria as a disease that’s relevant here?

Rajeev: Well, there are a couple of things to think about when you are looking at vaccine development. And one reason, and perhaps the most important, is to prevent the severe illness that comes from it. But there is another objective with malaria vaccines, which is blocking transmission. Now, we’ve all heard with COVID, that one of the goals of vaccines is to reduce transmission in the community to help us get a handle on the pandemic. The same concept applies when it comes to malaria, but the vaccine approach is very different.

And — this will blow your mind — when we think about blocking transmission of malaria, the way it’s being approached from a vaccine standpoint, is to prevent a mosquito from picking up malaria from a person that has it. And the way you do that is by designing a vaccine that will generate antibodies that are taken up by the mosquito, along with the parasites, and preventing the parasite from reproducing inside the mosquito.

So, you’re giving a person a vaccine that’s not going to prevent their illness, it’s going to prevent the parasite from reproducing in the mosquito, so you’re actually indirectly vaccinating the mosquito (and not the person), because that person could still get malaria illness.

Jorge: The fact that we’re vaccinating mosquitoes, I think is wild.

Sonal: That’s fascinating. Let’s actually talk about the findings.

Rajeev: So, basically what the researchers did is, they took this group of children that are in a place with a very, very high incidence of malaria, right before the malaria season, and they gave them three doses of vaccine with another dose a year later.

They then counted cases of malaria, so they monitored for fevers in the children, and when a child came in with fever, they would do a set of diagnostic studies, and if they were found to have malaria, then they would be classified as being a malaria case. And then you compare across the three groups, and the three groups were a control group that received the rabies vaccine, there was a low dose adjuvant group, and a high dose adjuvant group. And both of those had the same amount of the protein that makes up the vaccine.

Sonal: Can you quickly explain what an adjuvant is and why it matters, because everyone always mentions that and how that plays in with the way the vaccine works.

Rajeev: Adjuvants are what you might consider immune boosters. And so, for any given amount of let’s say protein that you’re giving somebody to train their immune system to recognize a virus or bacteria, or in this case, a parasite, you can get away with a smaller dose of that protein if you give somebody an immune-boosting adjuvant.

The adjuvant that is used in this trial is the same adjuvant that a company called Novavax is using for its COVID-19 vaccine. And this is an adjuvant that is chemically related to an adjuvant that is used by GSK in their vaccine against shingles that is currently on the market.

So, it’s an adjuvant that has been proven (at least in that vaccine) to be very efficacious. And at least based on the phase two data that we have, with the combination with this — what we call “virus-like particle” — it also appears to be quite effective at generating a protective immune response.

Jorge: I understand why you’d have a high dose and a low dose arm. Why is the control arm a rabies vaccine versus saline?

Sonal: Yes, I was wondering.

Rajeev: Yeah, well, you know, when you’re looking at the immediate safety of a vaccine — meaning sore arms, fevers, or chills that you might get after a vaccine — you’re gonna see that to some degree with many vaccines. And so, we want to do a “fair or appropriate comparison.”. And so, using another licensed vaccine that would be appropriate for the population that’s in the study is an approach that’s often taken to have the control group even out between.

Previous attempts and clinical trials

Sonal: I see. You’re sort of controlling for the variables you’re trying to measure. <Rajeev: That’s right> That makes a lot of sense. So, why — this is the real question here, the big-picture question — why has it been so hard to develop a vaccine for malaria? Now, we’ve talked already about the difficulties of the disease, but, like, if you look at the arc of it — Rajeev you said there’s one vaccine I think right, the GSK one, RTSS, and I think the last thing that I read was that they demonstrated 55.8% efficacy in African children.

Can you explain that vaccine, and tell us more about — I’m really trying to dig into, like, why it’s been so damn hard to actually get here, and why this milestone is so significant.

Rajeev: Well, it was a big deal when GSK showed that their malaria vaccine worked a few years ago in a phase three trial. Now they showed that initially they had about 56% efficacy in the first year. Unfortunately, that efficacy wore off over time. And so, if you looked at how efficacious it was after four years, it dropped down to 36%.

And so, that vaccine is not yet widely recommended. There’s a pilot program rolling out in a handful of countries that’s happening as we speak.

Sonal: And by the way when you describe the pilot program, you were involved with that organization, Gavi (I believe) is the one that’s sort of helping the World Health Organization pilot the GSK vaccine in Kenya, Ghana, Malawi, I believe.

Rajeev: Yeah previously, I served on the Gavi board, which is the primary financing entity for vaccines for low-income countries. The data coming out of that program will inform the future implementation of that vaccine. Based on that level of efficacy that was seen with that first malaria vaccine, the WHO later came out with a target efficacy of 75% for future malaria vaccines. That was what presumably these researchers were going for when they tested their vaccine, and they were actually able to hit that target.

Sonal: So, the GlaxoSmithKline vaccine has been through lots of clinical trials — a lot more clinical trials than this one has — and this particular vaccine, going back to this specific news, of the Oxford vaccine news. This is a phase 2B trial, it’s not phase three yet. Can you quickly explain what it means to be in phase 2b and what comes next in phase three?

Rajeev: Sure, sure. So when we take vaccines through clinical trials in people, we start out with the studies to assess the safety of the vaccine to pick up any significant problems in small numbers of people before you go into bigger trials. And that’s done typically in phase one. In those phase I trials, you’re also assessing what the right dose of the vaccine could be. So you might have high, medium, and low doses of the vaccine to give you a sense as to what the right dose is to take it to further clinical development.

In phase II development, you’re often going into [a] larger number of individuals, and confirming that you are at the right dose. You might even still have different dose levels in your phase two. In a standard phase two trial for infectious diseases, you’re not actually looking to see whether you’re preventing infection, because there’s a relatively small number of people in phase two trials, and that wouldn’t be enough to actually be able to statistically measure a difference in the vaccine group versus the control group.

A phase 2B trial is a little bit different: You have an even larger trial than a standard phase two, and these are often powered statistically to give you a sense as to whether the vaccine is actually working. This is a way that a company might de-risk the vaccine program before they go into a very large, very expensive, sometimes very long phase-three clinical trial.

So, this is that type of trial, where given the high incidence of malaria in this region and this population, they were actually able to show whether or not the vaccine works at preventing malaria.

The next step after this would be a large phase III trial. We can benchmark against GSK’s phase three trial of their malaria vaccine, which had about 15,000 children in it. And as we all know from COVID, the phase three clinical trials there have ranged from 20,000 to 60,000 individuals in any given phase three. So, this is smaller than that. And the main reason they were able to get away with a smaller trial is because the incidence of malaria was so high in the places where they were testing the vaccine.

Sonal: And for specifics, what I understand is that the recruitment of the phase three trial has already started, and they’re recruiting 4,800 children aged 5-36 months across four African countries. But here’s a little twist. So, you mentioned Novavax — so they’re one of the partners that’s collaborating with Jenner. But the other is, of course, the dominant player, the Serum Institute of India. They are obviously going through a massive COVID — so they’re actually delaying this a little bit, from what I understand.

Rajeev: Yeah, you know, it’s worth talking a little bit about that comparison between this vaccine and the GSK vaccine. Actually, they’re quite similar. They use similar adjuvants or immune boosters. The adjuvant used in the GSK vaccine is their proprietary adjuvant called AS01. In the case of the Oxford vaccine, it’s called Matrix-M, which is the Novavax vaccine. They’re both derived from tree bark (believe it or not) and so the chemical construct between the two, of the adjuvants, is quite similar.

The other parallel between these two vaccines is the virus-like particle approach is the same one taken between the two. They both use the hepatitis B surface antigen, which forms the core (or the base particle) for the vaccine.

The primary difference is that there is less of the hepatitis B surface antigen protein in the new Oxford vaccine. So there is proportionally much more of the malaria protein on the particle, than there is in the earlier GSK vaccine. And that is thought to be (potentially) a contributor to the difference that we’re seeing in efficacy with this vaccine, versus what GSK saw.

Sonal: And to be clear — I just want to kind of take the bottom line on that. Basically, what both of these vaccines are doing is targeting the parasite in that sporozoite phase of the lifecycle, which is when it enters the human body from the mosquito. And, the vaccines you’re saying — the main difference between the two vaccines — the R21 includes a higher concentration than the GSK. But that’s the primary difference. Other than that, they’re relatively similar underlying mechanisms.

Rajeev: That’s right, these two vaccines are quite similar insofar as they use a similar adjuvant, and they also have a similar structure.

Different vaccine approaches

Jorge: You know, in the case of this malaria vaccine, when we look at SARS-CoV-2/COVID as an example, the vaccine makers all sort of thought the spike protein was the best target or the most likely target, and that’s where the major players focused. Is there a similar consensus in malaria as to what the right targets are, or has that in and of itself been an odyssey?

Rajeev: Great question. There’s a lot of consensus around what’s called the circumsporozoite protein, let’s just say it’s CSP — which is the protein that is used in both the GSK RTSS vaccine as well as the Oxford R21 vaccine that we’re talking about today. And given that we’ve had limited efficacy with the vaccines against CSP — it’s not to say that something else won’t turn out to be better in future vaccine clinical trials.

Sonal: What happens if the bets we’re making with these vaccines — and this, I think, is the underlying thrust of Jorge’s question — is that it’s not actually effective at that sporozoite phase, do you have any thoughts on that? When we put all our bets on the coronavirus that hey we’re gonna focus on the spike protein, we’re taking a good bet — and so far it seems to have borne out, given that even with the new strains, that it’s still targeting the spike.

Rajeev: Well, it’s possible that we won’t be able to get there with this protein as the primary target, and that we would have to perhaps add a second target to the vaccine in the future. And it’s possible that we would have to add a second protein, or go after a different protein, in order to get efficacy against the parasite in this stage of its lifecycle. There are a lot of other proteins that one could target on this parasite.

Sonal: Got it. And actually, quick question for Jorge — is it naive of me to ask whether an mRNA approach would be more efficient? You guys put it really well, like, the old world/the traditional world versus a new world we’re in. Would a different approach to vaccines do a much better job? Because when I think of how we are thinking of these new batches of vaccines that we have in our bodies as, like, “software as a service,” are we able to do more with that — like, is that even on the horizon for malaria, or is that like just a pipe dream?

Jorge: I don’t think it’s a naïve question at all. You know, I — the vaccine producers that have mRNA technologies are looking at a broad range of infectious diseases for their next areas of focus. So, influenza is — you know, the flu is going to be an area of focus. The common cold is potentially, you know, on the table. So, viruses like that, clearly are sort of a next-horizon focus for these companies producing mRNA-based vaccines, and I don’t think it’s unreasonable to assume that their focus will expand beyond that.

I don’t know, technically, if an mRNA-based vaccine against malaria is feasible, but if you’re looking at surface proteins, arguably, you could theoretically develop an mRNA vaccine against the malaria parasite.

Rajeev: I agree with what Jorge said. I don’t think we can say that just because mRNA is a new technology that it’s more likely to be effective against malaria; however, I do think we can say that we need to give it a shot, because there’s so many advantages of mRNA approaches relative to more traditional approaches of developing vaccines.

One of the really interesting possibilities is that you could combine vaccine approaches into a single mRNA product. So, for example, you could have a CSP part of the sequence in your mRNA vaccine — which is the same protein that is targeted by these first two malaria vaccines that we’ve been discussing. And then you could have a second sequence or set of sequences that are targeting that sexual form of the parasite that the mosquito takes up when it has a blood meal, and then goes on to transmit the parasite to somebody else.

Remember, one thing that we need to realize is that second vaccine target where you’re trying to prevent the mosquito from going on to transmit the parasite — a vaccine like that would not prevent the actual illness associated with malaria. So, if you had a vaccine that was just focused on that, you’d be giving it to somebody and telling them “Look, this isn’t going to keep you from getting sick from malaria, it’s going to keep you from passing the malaria parasite onto somebody else.” That’s not a very attractive vaccine for someone to take. But if you combine that with something that also prevents them from getting sick in the first place — which could be this other part of the mRNA vaccine — then you’ve got a vaccine that everyone’s gonna want to take, and helps us to reduce malaria transmission, and maybe even eliminate malaria long term.

Measuring vaccine effectiveness

Sonal: Wow. Okay. In general, for context, over 100 malaria vaccine candidates have entered clinical trials in the past decades. But none has shown this level of efficacy that’s been targeted by the World Health Organization.

Now, again, to be clear, we’re talking about phase 2B. It hasn’t done large scale yet. But, one of the people who heads the World Health Organization malaria vaccine implementation program, argues that even modest efficacy would have a high impact precisely for the reason that people get the disease over and over again.

So, can we quickly talk about what the numbers of efficacy mean? They evaluated the vaccine safety, efficacy, etc., over one year — and what it means in, like, real-world practice?

Rajeev: Yeah, let me touch on the three parameters of vaccine performance that we often look at. One is immunogenicity. This is the easiest to measure. It’s simply the antibody response to the vaccine. We measure antibody levels in the bloodstream, and that becomes a measure of what we call immunogenicity. The reason that’s important is because the level of antibodies you generate with a vaccine might correlate to that vaccine’s ability to protect you from getting infected or contracting the illness associated with that infectious disease.

A second term we use in late-stage clinical trials is efficacy. And what that typically means or measures, is the ability of a vaccine to prevent the illness associated with an infectious disease. That term “efficacy” is very specific to the context of a clinical trial. So, there is usually a point estimate (let’s say 70% efficacious), and it’ll have a confidence bound around that, which represents kind of the error range given the size of the sample of your study.

The third measure we often talk about is effectiveness. Another way to think of this is real-world effectiveness. So, this is the assessment of how well the vaccine functions outside of a clinical trial, when you’re in the real world. You’ve launched the vaccine into a population (like we have with our COVID vaccines), and now we’re measuring how much illness and disease there is in a population.

Outside of the very controlled environment of a phase three clinical trial, where you may be telling people to do a number of things in order to protect themselves from the infectious disease. And so, you might see that a vaccine performs very well in the context of a phase three clinical trial. Then once you actually roll it out to the population, it doesn’t perform quite as well because these are real-world circumstances where people aren’t doing all the same things they wouldn’t be in the context of a clinical trial.

Sonal: So is there anything important to note about the question of effectiveness when it comes to this vaccine? Now again, it’s not in deployment yet, etc., but what are the considerations. One that of course comes to mind here is like, there’s like multiple doses.

Jorge: Multiple doses, yeah, I think that’s the biggest one.

Rajeev: That’s a big one. Multiple-dose is a big one. But there’s also, you know, in the clinical trial, people may have been very good about using bed nets at nighttime at home to prevent mosquitoes from biting. But in the real world, they may not keep up with their bed net use. They may not keep up with their indoor residual spraying of insecticides, which is often used to reduce biting in mosquitoes.

So, there are a variety of things that could increase the likelihood of getting infected in a real-world setting, which could correspondingly reduce your measured effectiveness.

Sonal: So, the other promise of this candidate vaccine from Oxford, is also the potential to get more high-volume, affordable vaccines. I read that it’s easier to make than the one that’s being used with the GlaxoSmithKline one, suggesting that it could be cheaper. Do you know if that’s true or not?

Rajeev: I’m not aware of the differences in the cost of making the two, because they’re quite similar. But it is using a pretty standard method of making vaccines. It’s manufactured in yeast (which is a tried-and-true way of manufacturing protein) — and the fact that Serum Institute of India has done this, means almost best-in-class in terms of efficiency, plus very high quality of vaccine manufacturing, given Serum Institute of India’s great track record here.

So, that’s also good news because this is a vaccine that is going to have to be priced at a level that is affordable for the poorest countries in the world, and can be purchased by Gavi. The fact that the cost structure is likely to be very low, helps to ensure that we’ll have a low price for this vaccine (at least for poor countries).

Possible next steps

Sonal: There hasn’t been a EUA for a vaccine in malaria, especially given the fact that malaria kills more people in Africa than COVID does currently. Do you think it’s possible that they might do some kind of EUA-type of situation for this?

Rajeev: I don’t think so. I would expect them to go through a standard (although accelerated) review process. One of the things that you are able to do in a standard review process is make sure that the manufacturing processes are all very well worked out, and validated, and reproducible with high quality.

It’s important for any vaccine, but certainly a vaccine like this that’s going to be going into vulnerable infants, you want to make sure that you do absolutely everything. And hopefully in the fastest time period possible.

Sonal: Right. I mean, we don’t even have a vaccine for kids for COVID yet, in fact, right, so.

Rajeev: That’s right, that’s right. One of the things that may be coming out of COVID is that we’re all a lot more attentive to global health problems that affect everybody in the world.

Jorge: I have to believe that we’ve also now developed capability, capacity, and political will in terms of vaccine production to do these things at the scale and speed necessary to hopefully benefit the entire world. My hope is that not only have we developed new technologies — you know we are now in a post-mRNA vaccine world (as Rajeev mentioned) — that have the potential to be um pointed at other infectious diseases, and hopefully give us other future breakthroughs.

When it comes to malaria, we’ve been pointing all of our guns at this for a long time. So, we’ve not only been looking for a vaccine, but as folks know, there have been philanthropic efforts to get bed nets out there. There’s of course tons of efforts in terms of insecticides, to reduce the population of mosquitoes. And there’s even, you know, engineering biology approaches to create genetically modified mosquitoes that are resistant to malaria. So, this is one weapon in what is a pretty deep armament to try to beat this thing.

Rajeev: Yeah, it’s absolutely true that this could be a critical tool in our toolkit. I look forward to seeing the peer-reviewed publication and hope that we’ll be seeing just as good results when the vaccine goes into phase three.

It’s also exciting to think about the end game. There is a day when we could imagine eliminating malaria from many more parts of the world, and possibly even eradicating it from the face of the Earth. Now, that’s not going to be easy, as smallpox and polio have proven (and certainly malaria is very different from those other diseases which are caused by viruses) — but it is something we can hope for.

Jorge: I was gonna say, we know the date of eradication will be April 25th, we just don’t know what year.

Sonal: Thank you so much, you guys, for joining this week’s episode of 16 Minutes.

Rajeev: Thanks a lot for having me, Sonal.

Jorge: Thank you, Sonal. Thank you, Rajeev.

• 

  Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

  Follow Twitter
• 

  Rajeev Venkayya
• 

  Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

  Follow Twitter

In this week’s episode of 16 Minutes, our show where we talk about tech trends in the news, what’s hype/ what’s real, and where we are on the long arc of innovation, the topic is semiconductors – specifically, the ongoing global shortage that began last summer and has intensified in recent weeks. So much so, that the U.S. president signed an executive order just last week to address concerns around the shortage, calling for reviews of supply chains for critical sectors of the economy.

Our expert is a16z Operating Partner Frank Chen, who led our research arm and has also joined past episodes about semiconductors on this show including one with Steven Sinofsky and Sonal in which they analyzed the ARM and Nvidia news.

Frank joins a16z’s Zoran Basich to cover the bigger picture of the chip shortage including geopolitics, the pandemic, and complex worldwide supply-chain dynamics — all in almost exactly 16 minutes!

Show Notes

• How the semiconductor supply chain works [1:14], what caused the current shortage [3:10], and errors in forecasting that some companies made [4:49]
• The impact of demand (types of chips needed by various industries) [5:54] and supply (cost of building factories, COVID disruptions, and the scarcity of raw materials) [8:24]
• Geopolitical questions involved [11:55], including competition with China [13:58]
• Thoughts about how the shortage will play out [15:14]

Transcript

Zoran: Welcome to this week’s episode of “16 Minutes,” our show where we talk about tech trends in the news, what’s hype, what’s real, and where we are in the long arc of innovation. The topic today is semiconductors, a topic that has been in the news a lot in different forms. Specifically, we discuss the ongoing global shortage that began last summer and has intensified in recent weeks. So much so that the U.S. president signed an executive order to address concerns around the shortage, calling for reviews of supply chains for critical sectors of the economy. 

Joining us as our expert is a16z Operating Partner Frank Chen, who led our research arm, and has also joined past episodes about semiconductors on the show, including one with Steven Sinofsky and Sonal, in which they analyzed the Arm and Nvidia news.

In this episode, we’ll cover the bigger picture of the chip shortage, including geopolitics, the pandemic, and several other factors, all in almost exactly 16 minutes. But first, Frank covers the immediate impact on the automotive industry, which was highlighted in the news of the executive order.

The semiconductor supply chain

Frank: A bunch of auto manufacturers, this includes GM, and Ford, and what used to be called Fiat Chrysler, now called Stellantis, all of these companies have basically shut down or slowed down their production of cars because they can’t get enough chips. There are very calibrated supply chains, tens of thousands of suppliers. And right now, what’s happening is, you know, Ford’s F-150, the best selling truck, is held hostage to a $0.25 cent semiconductor that all of a sudden they can’t get enough of.

Zoran: So this news is really closely tied to the supply chain. That’s where the White House is focusing. Help us understand the landscape here, Frank. Who are all the players involved up and down the chain and how are they interrelated?

Frank: The way that chips flow is that you have semiconductor vendors like Nvidia, and NXP, and Renesis, and Panasonic, Toshiba, Samsung, TI, and so on, they make the chips. Some of these companies make their own chips like Samsung and Toshiba, so they’re called integrated device manufacturers. Other companies like Qualcomm and Nvidia design the chips, but then use other companies like TSMC, or UMC, or Samsung, to actually make the chips. These design-only companies are called fabless chip companies. But in either case, you take the chips, you sell them to companies called tier one electronic systems providers. So these companies have names like Bosch, and Delphi, and Harman, and Denso, and Siemens, Continental. And then these companies in turn sell to what’s called the OEMs, the car manufacturers. That’s what we’d recognize as a car manufacturer. So a GM, a Ford, a Tesla, an Audi, a BMW, so on and so forth. Right? So that’s sort of the supply chain.

You got a chip company selling to an electronic systems company. They might make the onboard entertainment system, they might make the anti-lock brake computer, or they might make the adaptive cruise control system. And then the OEMs assemble them into cars. And so, there is a lot of complexity.

Causes of the current shortage

Zoran: So how did the automakers get into this mess? What are some of the factors that caused this chip shortage?

Frank: Let’s start with the demand side. So, three things. First, software is eating cars, which is to say that the percentage of a car that is electronics has been steadily increasing. So it used to be a car was rubber, plus glass, plus steel, and that was pretty much it. But now, in 2020, electronics are about 40% of the cost of a car. For a point of reference, in 2000, it was probably 18%. And, you know, these days, as you think about all of the cool safety features, right? Somebody’s in your blind zone, or adaptive cruise control, or automated stopping if there’s a pedestrian in front of you, right? All of those are obviously electronics.

The second thing is, this was a classic case of under-forecasting. So what happened was, COVID hit, and everybody battened down the hatches. Everybody was like, oh, car demand is going to drop off the cliff here. And so we better be conservative and cut back all our orders. And what happened roughly at the same time was, there was a set of things that people wanted that demand went the other way, right? So think of Chromebooks, and laptops, and webcams, and everything that makes Zoom possible. All of this work at home stuff created a lot of demand for TVs, and computers, and so on. And so, the demand for semiconductors to feed those things sort of leapt into that vacuum that the car guys left. And then once we figured out that car sales weren’t going to be as dramatically impacted by COVID, they went back to the suppliers, and then they discovered the supply is not available. In other words, they got spoken for by the computers, and the cell phones, and the webcams, and IoT devices.

Zoran: So there have been lots of headlines about this chip shortage, lots of angles to unpack, but let’s home in on the pandemic for a minute. How and why did this under-forecasting happen? And why specifically is that a problem when it comes to chips?

Frank: I think it was human nature. Which is, COVID happened and you were like, gee, when will demand really ever come back to normal? And I don’t want to be the guy that over-ordered everything, right? Like, once the TPS reports come out, I’m a complete outlier because what kind of idiot would I be, if COVID happened and I was the guy who over-ordered everything by two orders of magnitude. The other thing is, it’s very long lead time to spin up a new semiconductor, in other words, to change the line so that it’s making your chip versus somebody else’s chip. This is measured in tens to dozens of weeks, right? So, 30 weeks. And so, you can’t turn it off and turn it on, on a dime, right? And so, when the orders got cancelled, the lines got retooled to build other chips. And so, to turn the line back on to build your chip, we could be talking the better part of a year, which is why most people think that the shortage is going to be another couple quarters before we’re done with it.

Impact of increased demand

Zoran: Okay, so on the demand side, you’ve mentioned the rise of electronics and cars. We talked about the forecasting mistakes. Let’s get a bit more into what’s hype and what’s real here. What else do we need to know to make sense of this?

Frank: On the scale of things, car manufacturers aren’t the biggest customers for chips. So, by far, the biggest customer for chips is cell phone makers. Like, when Apple places an order, that’s an order, right? Like, that’s tens of millions, hundreds of millions, billions of components. And so, like, they’re more reliable as a customer. You can sort of see why if you are a chip manufacturer, like a Panasonic, or Toshiba, or Infineon, or STMicro, or TI — when Apple, or Samsung, or Dell, or HP comes to you and says, “I need more chips.” You say, “Of course. When do you need them?” Right? And then Ford, and Jaguar, and, you know, all of the other guys have to sort of kind of wait in line.

And then the other thing that is true about car chips is, they can be harder to make, which is, their temperature range, their operating lifetime, the failure rates of these chips, right? So it’s like one thing for your Chromebook’s light sensor to go bad, it’s another thing altogether if, like, the radar that powers your adaptive cruise control goes bad. Like, that thing can’t go bad, right? Because you’re going to crash into somebody.

Zoran: Yeah. And some of the chips that are used in cars are older generation chips. So, because they’re more expensive on the consumer electronics side, that’s another reason that they’re seen as more higher priority customers.

Frank: So, yeah, two classes of chips, the older stuff, right, for the anti-lock brakes, and then the newer stuff for what the car industry roughly calls ADAS, right? So, the advanced driver assistance systems. One way to measure this is, what’s the nanometer process technology used to create it? And we’re headed towards seven, six, five nanometers. Think of that as sort of how fine is the etching on a semiconductor that defines the circuit path, right? And car manufacturers — some of their chips, you don’t need [a] 7, or 6, or 5 nanometer, you’re fine at 180. But the second class of chips that the car companies are increasingly buying are basically the exact same chips that go into a smartphone core, right? So if you think about machine learning, as we head towards autonomous, you know, they will be the most advanced chips, with a ton of transistors on them to do linear algebra, because that’s what machine learning demands. And so, this sort of emerging class of chips are exactly the same set of chips.

Issues with global supply

Zoran: Okay, so that’s the demand side. When the pandemic hit, automakers pulled back on their orders, while at the same time, demand was rising for consumer electronics. And the chip manufacturers turned to that segment of their customer base, and started producing for them. And it’s very hard to stop on a dime, and then take the automakers’ calls, who now suddenly are calling you and saying, “Hey, we want to restart our production.” That all takes a long time. So, let’s talk about supply. What happened on that side of the equation?

Frank: So it turns out that the world has a fixed amount of manufacturing capability for semiconductors, which kind of seems bizarre, because we all know that software is eating the world. And, you know, the world has nearly infinite demand for chips. Most semiconductors are made outside of the United States, despite the fact that the United States dominates revenues for semiconductor design, through companies like Intel. But most of the fabs, which is the factories that make semiconductors, are overseas. But it turns out these fabs are incredibly expensive and relatively low margin to build. Think of a factory that might cost $10 billion to build, and it’s obsolete in five years. The rate at which semiconductor fabrication changes is so fast that, like, all of the equipment that you just bought from Applied Materials and JLA-Tencor, like, that thing is going to be obsolete in five years.

And so you have these incredibly expensive factories that depreciate very, very fast. The equipment in it sort of needs to get replenished very, very quickly. And so, the industry’s kind of rewarded companies that are called fabless designers. In other words, they’re companies that design chips, but don’t actually make them. So the fabless designers that we all know are companies like Qualcomm, and Broadcom, AMD, Nvidia, Apple itself, right — all of the, like, super awesome chips that they design, they don’t actually make. They go to semiconductor fabs, the largest of which is the Taiwan Semiconductor Company, that actually operates the factories. The top three countries that actually make chips, South Korea, Taiwan, and Japan, they have all of the factories because fabless is rewarded by the investing community. You’d much rather have the higher profit margins of a Qualcomm than the lower profit margins of a TSMC, the Taiwan Semiconductor Company.

Zoran: Okay, so you have this issue of how incredibly expensive it is and how frequently you have to accommodate the new technology they have to build. What are some of the other supply factors?

Frank: Well, like every other manufacturing facility, the semiconductor industry got hit by COVID itself. People need to be in the factories feeding the wafers, and doing quality control. And so there was some slowdown as a result of that. Now, the good news is, South Korea and Taiwan are the biggest manufacturers, and they were much less exposed to COVID because they were very aggressive with their lockdowns. But there are other things about semiconductor manufacturing. So, Taiwan is going through a drought, just like California went through a drought. And it turns out, you need a lot of water to make chips. TSMC’s daily water consumption is 156,000 tons a day. In the northern part of Taiwan, where these factories sit, that amount of water is 10% of the region’s daily supply of water.

And so, you have all of these weird supply things. You have, like, raw material shortages. We’ve got the COVID hit. We’ve got the very, very small number of fabs, just because they’re expensive, and sort of the financial community willingness to fund these super expensive factories is sort of low.

Geopolitics of semiconductors

Zoran: So we’ve talked about the factors like drought, the economics of chip production, the pandemic, obviously. Now, where do the geopolitics come in? How big a deal is that, really?

Frank: Yeah. So, geopolitics is a big deal. So, you’ll remember that the Trump administration basically forbid American companies from buying from Huawei. And then they later extended it to, you know, requiring a license to sell to Huawei. And then there are licenses that the semiconductor tool chain now has to apply for in order to sell to Chinese companies. And so, if there were no politics, you would do what manufacturing has done for the last four decades, which is, you’d fire up China. You’d make it possible for Chinese fabs, not Taiwanese ones, not South Korean ones, not Japanese ones, to bloom. But the problem there is that the United States, rightfully so, wants to be a little careful about what kind of semiconductor manufacturing equipment they will sell. Because the worry is that, they’ll buy it, they’ll reverse engineer it, they’ll infringe on the intellectual property. And lo and behold, they can make their own semiconductors. And by the way, that is China’s explicit goal, which is that they want to have the number one semiconductor design and manufacturing industry in the world by 2030.

And so, now the whole thing is a geopolitical dance. Like, you could imagine any administration, this is not a red or a blue issue, saying, like, we’re not sure that we want to sell China the equipment to make the most advanced semiconductors.

Zoran: It’s interesting because, you know, there’s this perfect storm that happened with the pandemic, and also some other factors, which we haven’t mentioned. You know, there were a couple of fires in Japanese factories that had a negative impact on supply. Even in Texas, there were a couple of factories that because of the recent cold spell, had to shut down for some time. So all these things kind of just built upon each other to create this somewhat perfect storm. And so that created this instant problem, but it also highlighted this larger problem [that] needs to be addressed. So what’s the bigger picture about what this means for innovation, given these geopolitical pressures?

Frank: Yeah. I mean, one of the big things that the industry is asking itself is, you know, China’s ambition to be number one, can they get there? And they don’t want to stop at just the chips, right? They want to be the complete, fully vertically integrated stack. So if you think of the vertically integrated stack, it’s sort of iOS running on top of a bunch of chips, with iOS. Or if you think about Windows, running on Intel. Like, in the next 10 years, will there be a Chinese operating system running on a Chinese chip, right, with all of the motherboard, etc., etc., design done by China. And if that were the case, will we really have two competing world ecosystems? It kind of reminds me of, like, the early days of communication. There was Docomo in Japan, and then there was Minitel in France. And like, they didn’t talk to each other. They were just little islands. And we knew that that world wasn’t good. What the world wanted was the internet, the connection of networks.

But if we go back to this sort of geopolitically motivated desire to have your country own the factors of production in a completely integrated vertical stack, hardware and software, then we might go back to the bad old days, where compatibility was hard. And, you know, we sort of were kind of wasting “R&D” dollars building the exact same thing, just in slightly different ways.

When the shortage may end

Zoran: So we have the White House, President Biden, calling for this review. Short term, what’s going to happen here? And how long will this last?

Frank: So most people are forecasting a couple more months, maybe quarters of pain. I don’t think, unless something surprising happens, I don’t think this is going to last that long.

Zoran: I thought the solution was more manufacturing facilities need to be built in order for this truly to become solved. So how is this going to solve in the short term?

Frank: You know, I don’t think it’s going to take, like, a brand new fab to unlock the current snarl that we’re in. So in the short term, you know, look, we’re not going to be at post-COVID highs on webcam orders forever, right? Like, they will go back to normal. And so the heat on alternative demand will sort of cool some. And then, you know, we’ve got the automaker chips in the queue now, right? And so, like, eventually, things will sort out. We’ve always had component shocks in the tech ecosystem. It’s just every now and then, we’ll hit a bad one. This one’s a pretty bad one because we have so many car factories making very, very expensive products, stymied by their $1 semiconductor being missing.

Zoran: And that brings us right back to the news, full circle. So, let’s go to our bottom line, Frank. What are your takeaways and final thoughts on the topic?

Frank: We have some soul searching to do about what the shape of our supply chains ought to look like and how much they should reflect the geopolitics of the time, or if technology wants to and should be a country-independent thing. Where, you know, the best ideas, meritocracy, take the day, as opposed to, we’re going to have the U.S.-led tech stack, and then a Chinese-led tech stack, and then they don’t really speak with each other, and view each other with mutual suspicion. The supply chain for technology has always had shocks. Because software is eating everything, these shocks are now rippling beyond technology.

Zoran: Frank, thanks so much for being with us today.

Frank: All right. Thanks, Zoran.

• 

  Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

  Follow Twitter
• 

  Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

  Follow Twitter

In this special “3x”-long episode of our (otherwise shortform) news analysis show 16 Minutes — past such 2-3X explainer episodes have covered section 230, Tiktok, GPT-3, the opioid crisis, more — we cover the SolarWinds hack, one of the largest (if not the largest!) publicly known hacks of all time… and the ripple effects are only now starting to be revealed. Just this week, the U.S. Cybersecurity and Infrastructure Security Agency shared (as reported in the Wall Street Journal) that approximately 30% of both private-sector and government victims linked to the hack had no direct connection to SolarWinds. So who was compromised, do they even know, can they even know?!

Because this hack is a supply-chain compromise involving various third-party software and services all connected together in a “chain of chains”, the knock-on effects of it will be revealed (or not!) for years to come. So what do companies — whether large enterprise, mid-sized startup, or small business — do? What actually happened, and when does the timeline really begin? While first publicly revealed in December 2020 — we first covered the news in episode #49 here when it first broke, and there have been countless headlines since (about early known government agency victims, company investigations, other tool investigations, debates over who and how and so on) — the hack actually began not just a few months but years earlier, involving early tests, legit domains, and a very long game.

We help cut through the headline fatigue of it all, tease apart what’s hype/ what’s real, and do an “anatomy of a hack” step-by-step teardown — the who, what, where, when, how; from the chess moves to technical details — in an in-depth yet accessible way with Sonal Chokshi in conversation with a16z expert and former CSO Joel de la Garza and outside expert Steven Adair, founder and president of Volexity. The information security firm (which specializes in incident response, digital forensics/ memory analysis, network monitoring, and more) not only posted guidance for responding to such attacks, but also an analysis based on working three separate incidents involving the SolarWinds hackers. But how did they know it was the same group? And why was it not quite the perfect crime?

image: Heliophysics Systems Observatory spacecraft characterize, in the highest cadence, the constant stream of particles exploding from the sun affect Earth, the planets, and beyond via NASA Goddard Space Flight Center / Flickr

Show Notes

• An overview of how SolarWinds was hacked [2:21], the attackers’ methods [5:33], and their impressive sophistication [8:17]
• A step-by-step explanation of how the attack took place [14:20] and how the hackers avoided detection [21:18]
• Open discussion of what the experts know so far [23:52], including how we know the attack was coordinated by the same group [29:21]
• Big picture security questions [33:26] and how businesses and consumers can protect themselves [42:51]

Transcript

Sonal: Hi, everyone. Welcome to this week’s episode of 16 Minutes, our short form show where we talk about the news, tech trends in the headlines, tease apart what’s hype/what’s real, and where we are on the long arc of innovation. I’m Sonal, and today’s episode is actually one of our special “2-3X” long explainer episodes — which I’ve done every so often for topics that keep coming up over and over in the news (most recently on Section 230 and content moderation, previously on TikTok, and even earlier on on the opioid crisis). You can catch all those at a16z.com/16Minutes. But today, we’re covering the SolarWinds hack, one of the largest (at least publicly known) hacks of all time.

Not only has it been in the news a lot since it was first publicly reported in December, with countless headlines since — but the most recent report, from the acting director of the Cybersecurity and Infrastructure Security Agency, was that approximately 30% of both private-sector and government victims linked to the hack had no direct connection to SolarWinds, as reported in the WSJ just yesterday. So, it’s gonna have ripple effects for quite some time.

So, we’re doing an “anatomy of a hack”: a teardown of the specifics we know so far, what went down, and what we need to know — whether big company, small company, or individual.

For quick context before I introduce our experts: Over 18,000 customers downloaded compromised software, though it goes well beyond them. Those customers include several large government agencies (which we covered last year on this show). Private sector victims include companies like Cisco, Intel, Microsoft, NVIDIA, Deloitte, VMware, Belkin, and others. The broad consensus – per a statement issued by the Office of the Director of National Intelligence, the FBI, Department of Homeland Security, and National Security Agency – is that Russia was most likely the origin of the hacking, and more specifically, that the Cozy Bear group (also known as APT29, overseen by Russia’s intelligence service) was responsible.

That’s just a super high level, because we’re actually gonna go deeper to break down the who what when how – and the chess game of it all. So now, let me quickly introduce our experts. Our in-house expert is a16z operating partner for security and former CSO, Joel de la Garza. And our special expert guest is Steven Adair, the president of Volexity, an information security firm that does incident response and forensics (including memory forensics), and they’ve responded to multiple cases of this. Their team actually put out several detailed posts on it and more.

Overview of the SolarWinds attack

Sonal: But first, Steven, can you summarize what happened? Obviously we’ll continue to dig in on the details throughout the episode, but the reason I’m asking is, I’ve started to lose track. I bet a lot of our listeners are getting a little inundated with this headline fatigue too, like — now this, now-what. So, tell me basically what actually happened. What do we know?

Steven: Yeah, sure. So, SolarWinds is a company that creates network and system management software that’s used really heavily by tens of thousands of organizations around the world, so it’s used by large giant commercial companies, Fortune 500. It’s used by small organizations, managed-service providers, and governments. So, it’s a piece of software used to manage these, like, really sensitive important assets. So, think about the IT teams, and people who wanna watch what’s going on key systems, on network devices, and things that are really important within a network. They have a product called Orion, that’s their flagship product.

And what happened is that SolarWinds was basically breached. How exactly, you know that’s not really been published. We don’t know. But attackers were able to compromise SolarWinds, get into what’s called, like, the “build process” of this product. So essentially, the development or the software that’s downloaded and used by all these organizations, they were able to get into SolarWinds’ networks, and modify that build process.

And what’s interesting and notable about this, is, they didn’t go in and modify the source code. What they did is — think about if you’re on an assembly line, and someone made a change like early on, and they all put it together — they actually waited til the very end, the very last step of compiling this package to make this software. It goes out. And they monitored it, they watched it, they looked at it, they learned, they tested. And they ended up compiling in a backdoor — which would give them access to the systems running SolarWinds Orion — for anyone who installed the update, or downloaded it freshly since they did this.

So, they were able to modify SolarWinds and push out this update to organizations all around the world. And basically, they’d create a shopping list and selectively target who it was that they wanted to go into, and basically break into and further their access. They could look through and see, “Oh, this company, or this government agency, I’m very interested in them.” They could actually activate and walk right into their network, and they’re already sitting in and going into a very sensitive part of that network.

So in short, it’s what’s called a “supply-chain compromise,” where, they’re really in the build process. Insert themselves to the backdoor into this legitimate software and expand their access — and do it very stealthily for many many months — until, you know, FireEye came forward and figured this all out in December 2020.

Sonal: Right. And just to quickly, even more high-level-context it — this is playing out against the broader landscape of — for many years now, companies have obviously been using various providers of third party and cloud software and services. We’ll delve into this whole notion of a supply-chain hack, what it means, what it means for the future of security.

But the thing I wanna really pull on from what you said is that this was very unusual because they didn’t go for the source code, they kind of waited for the updates, and then they were very targeted — as opposed to just sort of spray and pray. So, in your assessment of all the hacks that you’ve seen out there — and Joel I wanna hear your thoughts too here — is this really a sophisticated hack? Because obviously, in our show, we not only tease apart what’s hype/what’s real. I often wonder if that word gets thrown about very casually.

Steven: Yeah, so from our opinion, it’s definitely — this aspect of it — is certainly one of the more sophisticated that we’ve seen. And it’s not necessarily that there aren’t a lot of smart people around the world, good and bad, that couldn’t pull off something similar. It’s, you know (1), the fact that they did; (2), they did it so strategically; and (3), you know, even if they had gone in and modified the source code, people would still be talking about how sophisticated it was. But, they took it up a notch and basically said, “Yeah, we modify this code, or someone’s watching it, or they audit it, or someone’s watching a check-in process.” Basically it went to a system where none of that mattered anymore. And they just kind of bypassed all that and went, like, straight for the jugular, in what would — I would argue a much more difficult way to go about it, but a lot more likely to meet with success and go undetected in that. I think they gambled correctly in this case.

Joel: I mean, I think with these kinds of operations — and this is ultimately an espionage, you know, nation-state professional-type operation — from my perspective, the duration and the extent to which these things can run undetected is usually the indicator of how sophisticated they are. And so, like, these long running, you know, really successful campaigns that avoid detection really belies like a level of sophistication.

Because operational security, right — like, covering your tracks — is actually just about as hard as getting in. And so, you know, the fact that they exercised their ability to cover their tracks for so long, to know where to insert in the process, and to lay low, is just indicative of a level of discipline that you don’t necessarily see in a lot of attackers.

Sonal: Not just get in, but be able to cover their tracks, which is what both of you guys say. And by the way, we’ve only talked about the duration of when the hack was revealed by FireEye, and that it had been you know several months before. Do you guys have specifics on what the latest date-point is, in that timeline?

Steven: Yeah, at first, essentially what they did was an experiment early on — and this has been posted publicly. The code in SolarWinds Orion was modified in late 2019. Where basically they made some initial modifications, which actually didn’t do anything malicious or put a backdoor, allow any type of access.

Software went out. And they basically were able to prove, like, “Hey, I succeeded at doing this, it existed, no one noticed anything” — and essentially waited at some point to move on to phase two, which was, “Okay I can get in and go undetected, I can have it build, it all works, stuff makes it into production, no one notices.” And they said, “Okay, well, I’m satisfied with that. Now it’s time to, you know, go for broke and put the actual code in there, and open the floodgates.”

Sonal: What you just described, Steven, sounds exactly the way a company builds a product. Like, “Hey, we’re gonna test it out. We’re gonna try an experiment, an MVP, a minimum viable product, if you will. Then we’ll, based on that, decide how to deploy it and target it, and blah blah blah.” I mean, I hate to say that, but that’s exactly what you just described sounded like.

Steven: Yeah, it honestly wouldn’t surprise me if they had done some way of trying to basically clone their development environment too, and probably tested this — I would guess — probably pretty thoroughly before they even <Sonal: wow> ran the tests within their network

The hackers’ sophisticated methods

Sonal: So, they were incredibly savvy in certain ways, in terms of how targeted they were, and the choices they made.

In the Microsoft blog post, one line in particular really struck me. It said that the threat actors were savvy enough to avoid giveaway terminology like backdoor, keylogger, etc. Instead, they gave their tampered code an innocuous name, “Orion Improvement Business Layer,” that would fit right into a marketing brochure. (This is from an Axios post summarizing it.) “The attack’s crucial door-opening exploit was a small chunk of ‘poisoned code’” — which is what Microsoft dubbed it – “all of five lines long or roughly 160 characters.” And then Ina Fried at Axios goes on to comment (which I had to chuckle, even though it’s sad), was, “This could well be the most damage per character yet achieved in the short history of cyber warfare.”

So, I am curious if you have any thoughts on some of those — honestly quite clever — things that they did, to hide undetected. And any more specifics you could share there. And then we’ll go into the step-by-step in a moment, too.

Joel: The fact that they’re not naming variables and naming things that are commonly used in attacks is mostly a credit to the existing kind of antivirus and anti-malware industry. You’ve got a lot of tools that are out there that are looking for this stuff. And you would imagine any adversary that’s relatively sophisticated is gonna run their changes through all those tools to make sure they don’t get detected before they deploy it.

And so, that’s just table stakes for this kind of activity. It doesn’t really show any kind of real sophistication.

Sonal: Of course, it just depresses me to hear that — and we’ll talk about this at the end, which is what companies and people can do. Because I’m, like, great — the better and better we get, the more and more sophisticated they get, and it just becomes this like never-ending back-and-forth, back-and-forth escalation.

Joel: Espionage 101.

Steven: Yeah. To be completely honest, that stuff doesn’t surprise, especially when their job is to, like, blend in as much as possible.

But I’ll add to one of the things — and make sure that we give credit — some of the analysis of things we’re talking about today are obviously from — a lot of security communities have come together and published a lot of detail, which has been great. But this is one of the other things that they did, is, they actually used an existing config file that is part of SolarWinds Orion, that’s there legitimately — it was there five years ago, it was there two years ago, it’s there right now — but they actually repurposed that exact config file. They created a specific value and said, if this is a three, you shouldn’t beacon it, you’re basically turned off. And they use values in fields within this to then leverage that file that’s already being read and used by the program, to then also inform it on some of what it should do.

So they use, like, native, existing files and functionality and things that are very innocuous-looking. And then they did a couple of other stuff beyond that, that are pretty stealthy –– although they’re not necessarily rocket science, they are very uncommon.

~ One of them is the fact that this backdoor, once it’s loaded, it wouldn’t start its beaconing or calling out for this DNS activity (which I know we haven’t explained yet), but basically, the mechanism by which it actually gives that avenue of control back into the system. You have to meet certain criteria before [you can] even, you know, beacon. For example, if you weren’t “domain joined” — meaning you’re less likely to be an actual corporate asset. You’re someone testing it on a computer, you’re a workstation at home. You’re not even gonna pass the sniff test.

~ But what they then do is actually set a timer. And so, it might be actually up to two weeks before it actually starts doing anything. I might be under scrutiny from QA, or a build, or someone might be looking at it when they first install it, make sure it’s not malicious — so they actually say, “Hey, I’m just gonna wait two weeks. I’m in this environment, this is for the long haul, I’m not in a rush to immediately get access to these systems.” So that’s an interesting aspect. It’s actually fairly uncommon to see malware that is on any timer of significance, or driven by a specific event that’s likely to happen very soon.

~ The other thing that was really interesting: The malware basically would activate when a certain response was given to its query. “Hey, go connect to this domain name,” or “go connect to this website.” And, those domains that they used were actually domains that had expired. One of the telltale signs when you’re looking into malware and things is, like, “Oh, it was just registered last week or last month or earlier today.” So, this would pass that sniff test, all day long. Some of them had five or six years they had existed. It might even have, like, a website. They picked up infrastructure that had a history to it. They actually owned and controlled these domains. They weren’t, like, hacked domains or things like that, where they were using compromised infrastructure. So, just kind of an interesting note on that front.

Sonal: It’s interesting and, honestly, a little creepy. I got goosebumps while you were talking, because it makes me think of every long game. The patience, and waiting, and stalking — that really skilled predators do. And I don’t mean to glorify it by any means, but I am just sharing that what you just shared in technical terms — it gave me goosebumps, quite literally. I don’t know how you think about it.

Steven: When we first saw this in July of last year, we had I think three domains that we had seen used in that actual attack. And as we looked into them, we said wow. Like, we kind of noticed it’s just, like, yeah, these things have a real history. You know, what the hell is going on here? And then we found a way to find more of their infrastructure (even if we hadn’t seen it used in the attack), and they all had this in common. Like, we had a way which we could figure out and find some infrastructure from some mistakes that they had made. That’s why in our post we actually were able to provide a lot of indicators. Like, DHS included that in their list and everything.

But, other than that, each one of the domains we looked into, we just instantly knew at that point — I mean, we already knew we were dealing with an advanced threat actor, but — we were kind of thinking to ourselves, like these guys have really stepped it up a notch. This was actually the third time we had dealt with them in an incident-response engagement. But this was, like, a little bit different than the other two rounds. There’s a number of things that just made it stand out, and that was definitely one of them.

Sonal: This might be the first a16z Podcast Network show to be optioned for a movie. I’m just gonna say it right here, on air. Joel, anything to add to that before I switch into the detailed step-by-step?

Joel: I mean, only if Matthew McConaughey plays me. <Sonal laughs> No, I’m just kidding.

Sonal: I listen to him on the Calm app every other night or so.

Joel: Yeah no, I mean I think that’s exactly it. Just the level of preparation, and just the long game that these guys are playing.

You know, this malware stuff is pretty common on the financial crime-ware type side, right, people trying to steal money. But those actors typically register domain names within a day, it’s just all very phish-y and suspicious. But to see someone build these, like, really advanced, large, complicated infrastructures, years ahead of using it — it just belies a real level of sophistication, you don’t really see every day.

How the attack took place

Sonal: Okay. So, just to recap for listeners where we are and where we’re going — we’ve covered what happened at a high level, including some of what’s hype/what’s real, and interesting or undercovered in the media.

You did a great job summarizing, Steven, but let’s now spiral into that a bit deeper and fill in some blanks that you haven’t covered. Both technical details — you mentioned the beacon, DNS — I want all of it. How folks figured things out — so we can then know what the open questions still are, ripple effects and implications, and then more on supply-chain compromises and what we can all do. But I especially want to know the anatomy of how they got access to the emails. But start from the very beginning of the timeline.

Steven: Yeah, so the story of the SolarWind supply-chain compromise obviously starts with SolarWinds — and that’s probably where some of the question marks are currently, and they might remain that way. They were breached sometime at least as of late 2019, and then ultimately — what came out later in May of 2020 — pushed out an actual backdoored version of their software. A backdoor meaning, a piece of software that shouldn’t be there, that allows this foreign adversary to have control or remote access into these systems. So we’re talking in late May, that happened. From the cases we’ve been involved in and things that have been published publicly, we’re seeing that a lot of the threat activity started in June and July.

The SolarWind software would send out this DNS query. So, when you want to go to a website, you wanna go to a16z.com, you type that in. There’s a system called DNS, it says, “Hey, where is this located?” A DNS server says, “Oh, it’s located over here.” It’s the basis [through] which kind of you can find things on the internet, so you’re not memorizing these numeric IP addresses. 

So, the malware — all it did, once it finally activated — it waited between 10 and 14 days before it would start creating these DNS queries — it would do these DNS queries from the SolarWinds Orion server. And those DNS queries contained encoded data. And if you decoded that data, it gave you different information, but one was information about the network that that machine is joined to. So for — in the example of, you know, say, Microsoft, it might show Microsoft.com or Microsoft.internal. Or, you know, one of these government agencies, it might say treas.gov.

But it would give this indicator, so that the attackers could actually see who these victims were — because remember, they were indiscriminately pushing out this software, potentially tens of thousands machines. That is an untenable thing to manage, and go and manually look at everything, and try and actually install software and do something of significance. And their goal is to stay under the radar, and not get caught. And then now they have to decide who it is they want to go after further.

So, they probably have a shopping list that they started with, and they probably have a new shopping list of things — they’re walking into the grocery store and didn’t even know they wanted that, but now they know they do. And they essentially issued commands, and allowed them to initiate this backdoor on who it was that they wanted to attack. And they did this through a specific DNS response called a C-name value. So, it says, “Hey, where’s this host name?” It responds back. They would actually send a specific response to prep it, so that the malware would be waiting to know that next time something happens that it should take a specific action and open the backdoor.

It would respond with these domains. And these domains would basically be the control points of where the attackers within have the hands-on keyboard — a human is doing this at this point. Someone says, “I am ready to take a look at this system,” and now hackers that are behind this are actually involved, and they’re saying, “Now I wanna look around and figure out. Is this a test machine? Is this a real network I’m interested in? Is this a lab environment? Is this a staging environment?” You know, things like that. And they can figure out, “Is this the real deal? Does this have access where I want? Do I want to proceed?”

And they did this for — we don’t know how many organizations, and that’s the real scary part in all this, is — you have all these people that have come forward, and they’re, like, big companies or they’re these government agencies, and, that’s just the ones we know about. I don’t think anyone has a real notion of the size and scope of where they took a further interest and then actually did something. In our particular case, we got permission to write up and share details of our incident investigation. The attackers were very focused on getting access to email of specific individuals. So, their goal was maintain access, move around — you know, get what they need — having access to specific individuals, and what they’re writing, who’s sending them, why they’re communicating — was a key focus of what they’re doing. We were able to see that they did that.

The interesting part, in kind of stepping away slightly from SolarWinds — and why the intel community and law enforcement says it’s likely tied to Russia (APT29, or the Dukes) — we’ve been tracking a group we call Dark Halo, just because we’ve dealt with APT29 on many occasions in the past, but we just have no real way to link the two.

But what was interesting to us, is the story of this group didn’t start with SolarWinds. We worked three separate incidents involving these SolarWinds attackers, who we called Dark Halo — so, this is a story that starts well before, and has multiple other avenues.

We had actually dealt with them back in 2019. We had an organization we were doing work with, and we kicked the group out. They went away. In our initial response, we had determined they’d been in that organization for 4-5 years prior. They came back in Q1 2020 through an Exchange control panel vulnerability. You know, mail service — they had a vulnerability that attackers will take advantage of. Got back in, stole email for certain individuals. They were kicked out and removed again. That’s what we did. And then they came back a third time with SolarWinds in July of 2020 again. We didn’t have a good way to prove it, and we took steps and mitigations in place to deal with it.

So to say, “Hey, how did they get into, you know, SolarWinds,” or wherever else they’re operating — well, this isn’t their only trick. They have a lot of tricks up their sleeve, and they’ve been able to do this and operate for quite some time.

Sonal: Wait, so how did you make that link across those separate incidents that it was the same group?

Steven: I’ll tell you, and it was something interesting, is — if we had worked them at three different organizations, we actually wouldn’t have come to the conclusion that this was a single threat group. We wouldn’t have linked the three things.

Any advanced attacker, anyone in network, they have certain commands and things that they’re gonna do — but they changed enough between each of the attacks, that the actual techniques, the tools — there’s a custom malware, or a commercial script, or a public script, like <inaudible> or a pin-testing framework, or these different toolings, or a web shell — they changed it between each one of the hacks, where it was able to be very non-obvious it’s the same group.

But what they did is they went after the email of the same people each time — and why we are 100% certain it’s the same group, is — when they would steal email. They would only take a certain amount of email. They would specify, “I want all the emails since the last time I took it.”

Sonal: Oh, so it’s like incrementally building on the total — oh my God, that’s so fascinating. <Steven: Exactly!> Keep going, yes.

Steven: So in early 2020 they got back in, and they said, “Okay, well, I want all the email for these particular users since, you know, a specific date in 2019.” And then when they came back in through the SolarWinds vulnerability, they basically said, “Hey, I want every email for these people, and I only want it starting from this specific date range starting in early 2020.”

So, we had each time they came back and asked for the email since the last time they did it. So in the one case, obviously, they had an intimate and previous knowledge. The other cases we worked, they didn’t have as much knowledge. They had to work their way and kind of figure out the way of the land. So, we’re dealing with the same group in all three incidents — that’s an interesting tidbit.

Sonal: I was about to say, I still have goosebumps. That’s incredible. That was so good, Steven.

How the hackers covered their tracks

Joel: Pretty impressive analysis and work there.

The things that really jump out to me is, this is something that is linked together over a 4-plus year campaign, trying to maintain persistent access to the communications of high-value individuals.

I think the other thing that really jumps out to me is that they have a big data problem. They got access to tens of thousands of computers, and potentially thousands of organizations. It sounds like the kind of analysis that Steven has done is pretty unique. There aren’t a whole lot of people in the world that can do that sort of thing. And so, this is probably an incident that we’ll be continuing to understand for the coming months, if not maybe years.

There’s probably gonna be a really long tail on that. These people are still out there, they’re still operating. What are they doing now? That’s particularly concerning.

Sonal: It’s interesting because Martin Casado — you know, our general partner, who’s also a security expert — he mentioned to me that he thinks it’s super interesting how interactive the attackers are during the attack. Because it’s obviously a very sophisticated team of people gathering data and making chess moves in real time.

And it’s so fascinating because when we report and talk about and communicate these types of attacks, we kind of make it seem like it’s malware that does all the work — but it’s really the people that are at the center of it. And then on the other side of it, you have this whole interesting dance, on your end — as sort of this forensics expert with your team, going in, and trying to figure it out, and the puzzles, and everything involved.

Joel: Well, you know, I heard chess is popular now.

Sonal: <laughs> Queen’s Gambit, right.

Joel: This is exactly like playing a game of chess. The difference is that you don’t see the moves immediately — they get revealed over time, and then you’re left kind of piecing other things together.

Sonal: That’s exactly the analogy.

Steven: Yeah, I definitely agree — that their goal was to actually not have their moves — what they did never be understood. You know, we noticed the versions of their software that were downloaded. There was an update to SolarWinds Orion — I believe it was in August of 2020 — and that version wasn’t backdoored anymore. It didn’t have the malicious code. So we initially speculated, “Oh, did the bad guys remove it? Did SolarWinds find it? Did it inadvertently get removed?” We didn’t know how it was going down at the time.

So, they removed the code. They got in, got all this access, and basically said I’m gonna try and remove this now and, like, fly under the radar. So, if they had their way, they would have pulled off like the perfect caper, done all this stuff — no one would have known how it happened. And then the Orion product, basically, would have nothing malicious in it. <Sonal: wow> So, just a, kind of like an interesting other thing that they did.

Sonal: It is. It’s a very vivid contrast to the analogy of chess, especially given the popularity of Queen’s Gambit, when you see them recording their moves, and the spectators watching — it’s a real contrast to this idea that you’re literally making the move, peeling it back, making the move, peeling it back — it’s really stunning.

Open questions for the experts

Okay. So my next question before we talk about some things we can expect to see moving forward — what are some of the open questions still on the table? Like, we know SolarWinds was compromised, but the big open question there is obviously we don’t know how. Then the second big thing in the Microsoft post that I saw (and Steven Sinofsky pointed this out), which is, you know, they do this outline, but we still don’t know how the signed code was signed, so that whole idea of “sign the code” is a bit of a mystery still.

I want to hear from you guys, what are your open questions — or what are the open questions the industry is still looking at, or that people should or shouldn’t look at?

Steven: Sure, yeah. So, how was SolarWinds compromised? Obviously one of the open questions. You could spend as much time and resources — you could use infinite resources, and you may not ever be able to answer that question because that system is gone. It was wiped. All the logs are here, that was never logged, or it happened five years ago. So, I would say the scariest part of this — people are finding out about this in December, for something that was operationally live in May. They had a looong headway into breaking into different organizations, doing that shopping list. And there are going to be — and there have been — from this very group, and as a result of the SolarWinds compromise, more supply-chain breaches.

Some people are breathing a sigh of relief, “Ahh! I didn’t run, you know, SolarWinds Orion software. I’m safe.” That’s not necessarily true. We’re not trying to sow fear, uncertainty, and doubt that everything is untrusted — which arguably, you need to go to a typewriter, send pigeons now — but it’s IT companies, it’s security companies, it’s managed service providers, it’s managed security service provider. There’s these different people that were running SolarWinds that then had this level of access to either directly get into networks, get into email, get into authentication systems to provide software or software updates or software downloads. They 100% certain had access to numerous networks and systems that would allow them to rinse-and-repeat SolarWinds, probably on numerous different scales, in numerous different ways. It doesn’t have to be through a build-time compile. It could be, they change a download, they change an update process. They took keys, or secrets, or remote access protocols, or passwords that got them into like other networks or other systems.

So, the scary part is, is that the supply-chain compromise here is just causing a chain reaction that’s probably already impacting other organizations that have no idea. I think that’s one of the biggest questions, is — who else was victimized that we don’t know about, and what do they do?

Sonal: So what you’re basically describing is, like, this complex, adaptive system — like, everyone sort of networked and connected trying to tease apart the scope and ripples of this is gonna take ages. And we might never, ever get to the bottom of all of that, because of that connectivity.

It’s interesting because General Paul Nakasone, or Nakasone — I’m not quite sure how to pronounce it — he heads both the NSA, the National Security Agency, and the military’s U.S. Cyber Command. One of the things that they talked about is that developing a coherent, unified picture — what you just described, Steven, of the extent of the breaches — has been difficult. The challenge is that, “He’s expected to know how all the dots are connected, but he doesn’t know how many dots there are, or where they all are” — which is kind of a distillation of what you just described. What are the other open questions that are on the table?

Joel: For me, the big open question — and with all of these really sophisticated breaches, the first is, how many stupid things led up to this? Like, how many ridiculously, easy-to-solve problems, like applying security patches, or using two-factor authentication — like, how many of those kinds of things we know we should always do are responsible for this — is always front of mind when we see this.

Because I think when you double-click on these, a lot of the times it starts off in a fairly innocuous way, which is, like, someone guessed an account, or someone got access to some account. But as this event shows you, if you give a sophisticated actor a toehold in your organization, they’re just gonna run through it. So, that’s the first one.

And then the second one is, we think of these breaches — because of just the way the media covers them, and the fact that they kind of show up sporadically — we think of them as, like, events in time that have a start and finish. But, in reality, these groups are still running, and we’re still facing them. You don’t know the implications of any of this stuff for a while. Like, you don’t know if they were getting into the Department of Energy to read, you know, Rick Perry’s old emails, or if they were getting in there to steal futuristic bomb designs. Maybe there’s gonna be some new weapon that pops up in 15 years and it’s, like, linked to this breach. And we’ve seen from these breaches — like, if you go all the way back to some of the first ones that have been publicly reported — you know we’ve often seen that the goal of these is either to spy on individuals and get some intelligence there, or to steal the designs for things that people want to go recreate.

Sonal: Right. And don’t forget that oftentimes — I think we often forget to talk about, when we talk about intelligence — it’s often in the form of blackmail, right? Like, we’re not just talking about stealing IP and obvious secrets.

Because a lot of people dismiss this as, “Oh, email. I just book events and share, like, photos with the family in my email.” I don’t think they realize that it’s such a vector to all these ways of really exposing who you are. It’s your identity, in many ways. So, that’s another way to think about that too.

Joel: Absolutely.

Sonal: Anything else on the open question side?

Joel: So, a bunch of other secondary breaches are now being reported on. Some of the Microsoft stuff, you saw that there were people creating reseller accounts, or trying to get reseller access to people’s Office 365 enterprises. And then there were certificates that were compromised for things like Mimecast, and maybe perhaps other services that are out there.

And so, like, this picture starts to emerge that there’s these — lots of fires just started burning. And it’s always really difficult to tell if it’s one fire massing together, or just a bunch of different people that are acting independently.

Sonal: That’s actually something I wanted to really quickly touch on before we go into the rest of this. Because the thing that was confusing to me is, okay — so, I read the Microsoft post. You know, like, there’s some intrusions. That there was a partner for Microsoft, actually, that handles cloud access services. We don’t know how connected or not connected it is. Then you have a reseller gaining access to Microsoft customers’ Azure accounts. Then you have this reported Russian state-sponsored effort exploiting a VMware flaw that the NSA warned about last month, that takes advantage of a recently announced vulnerability in VMware Workspace One access. Access Connector, Identity Manager, etc. And, this is according to the NSA, that they’ve had at least one case — that they’ve successfully accessed protective systems by exploiting the flaw.

And then you have, like, you know, one after another, and they issued a patch. I mean, I am reading all these at the same time and I’m like, is it all the same thing or not? I think that’s what you’re saying, Joel, about — we don’t know if it’s all one fire, or a bunch of fires. And do you guys have any thoughts on how to connect those dots, if at all?

Steven: So, as a general statement, I would say what we know about this attacker that we call Dark Halo — the people behind the SolarWinds hacker — they’re extremely adept in methods that allow them to gain access to email or systems involved with email. So, things like trying to get access to an Office 365 or Azure AD environment through a partner organization. Or by stealing some, you know, SAML tokens or some kind of authentication mechanism. Or, trying to get access through some other — possibly through a vendor — to get access to that same data to email data, essentially by any means necessary. I would say all of those are very on par with what we’ve seen this attacker do and focus on, and what others have seen. A very good chance that they are related.

But even if they weren’t, it just kind of underscores that there’s a lot of people trying to get access to this data. And now you need to focus a lot more on the cloud, on the technologies that are used to secure the cloud or that have access into it. And the things and places where people don’t always look — because it’s new to them, or they never looked at it, or they didn’t know to look at it — so, I think this event will actually end up advancing security in many ways, because it’s causing people to think about and do things that they weren’t realizing before. And as you can see, the bar’s been set higher to where they can’t walk right in the front door anymore, right? They’re not easily able to get right into these organizations by compromising, you know, the core network or the system administrator and the other ways which you could get there.

So, in some ways, it’s a sign that security has improved a lot — but also that there’s a massive amount of work to do at the same time.

Sonal: It makes me again think of the chess analogy, and when you have a player that comes to the table that has a set of moves — like, patterns that are well beyond what the human mind can even comprehend — and that makes me think a little bit of even, like, AlphaGo playing Go with a real chess player in Korea. And how you know the system made moves that they considered very alien, but that a human being would never have done, but that still follow the rules of the game — the constraints of the game, that is — and yet were completely novel. And if you just keep seeing more and more moves kind of grow and become more and more sophisticated on both sides — even as we may improve, like, there are gonna be alien moves at some point.

Steven: Well, to be completely honest, they’re undoubtedly highly skilled and disciplined — which, if you think about it — okay, if we go back to the chess analogy. You know, are they a master, are they a grandmaster? In some ways you can say, okay, they’re a grandmaster — but most of their opponents are unranked. So, they have this kind of lower skill, and their strategy is easier. But then they’ve been able to go to these people who maybe their security defenses are much higher ranked, and they’re using that skill set, that knowledge, and that kind of cat-and-mouse, to still get into those organizations. But to have to do that, it shows that people have leveled up quite a bit — which is a good thing for these companies and the security industry.

But at the end of the day, they still managed to either capture that king or get them to knock it down. I guess no one’s really thrown in the towel. No one has surrendered, that I’ve seen so far. But, I would say they’re winning a lot of matches, and they’re playing a lot of them simultaneously.

Sonal: Right. But they are not (to be clear, to your point), an alien player, like an AlphaGo. They’re still moves that are human, just very skilled.

Steven: At least from what we’ve seen, but who knows, like, what we’re missing though, right?

Broader security trends

Sonal: Right. Okay, so now the big picture questions. We’ve covered what happened, how it happened, the details. We talked about this, you know, phenomenon of supply chain attacks, chain-of-chains, what it means. I would love to hear what you think about this, when you think about the broader trends at play.

Joel: Yeah, absolutely. I think on the podcast several times — I know I sound a bit like a broken record — but we’ve talked about the biggest challenge being securing the supply chain. And how all these businesses that are becoming software businesses are actually becoming reliant on other people’s software. And so, it’s not just a matter of the stuff that you write to run your company, it’s also the matter of the stuff that your suppliers are writing.

And as everyone knows, security is really difficult, and it’s hard to secure your own things — and then having to worry about the security of your suppliers is adding an additional layer of complexity.

And so, over the last couple of years, there’s been a lot of investment in trying to understand third-party risk management, vendor-risk management. How to glue these things together. There are several different approaches, everything from private systems that will look for vulnerabilities and report on the risk. There are publicly available standards, different trade groups are trying to develop their own standards for security — and then certain vendors are trying to come up with their own standards. There is no easy answer, and so what you’ve got is a lot of different approaches that are being tried, and a lot of experimentation that’s taking place. This is probably the first breach at such a size, scale, and scope. So this is kind of the watershed moment for that third-party risk management.

And there’s any number of other suppliers that are out there that are in very similar positions, right, and it could be a company like SolarWinds, or it could be an open-source repository that a bunch of people are building into their applications. There are any number of different ways.

The thing that’s really difficult for me — based on where I sit and what I see — is if you play through all the different potential solutions that are out there, it’s really hard to know which one of them would have actually prevented this? So, like, if I went to any of SolarWinds’ customers and said, “Hey, what’s your vendor risk-review report on SolarWinds?” You know, before the breach, I’m sure they would have said it was a wonderful company, it was doing everything, they passed our review, they answered our questionnaire. You know, they’ve got the people hired, they have a program. And so, it really comes down to how do you actually measure these things, and how do you measure the risk in that third party, and how do you effectively mitigate against it?

Steven: The third-party risk or the vendor-risk management or how that someone evaluates this — it can only go so far, right? Like, how would you evaluate SolarWinds and the Orion product any differently than you would Microsoft Windows and Defender and how it updates and things like that, right? So there’s limitations to what you can do. I mean, you could audit them, or find out their code-review process and all that stuff — and they could have passed that all with flying colors. Or does your checklist say, “Are you looking for advanced adversaries, you know, injecting themselves into your build process at the highest levels of sophistication and espionage?” But even if they check yes to that, which they might not, they probably aren’t having an effective way or mechanism to do that.

Sonal: One of the things that Alex Stamos — people tend to over-quote him, but he did have a good tweet about this, which is — “There is no good reason for most enterprise software products to talk to random internet hosts all day. It might be time to move on to an outbound network-permission model for Windows servers, so connections only allowed to domains and signed manifest plus internet as defined in GPO.” Is that the right thing to do? Should people be air-gapping? Like what should people be doing?

Steven: We deal with sophisticated breaches all the time, and this can even apply for, like, crimeware and other stuff, but that is a recommendation that Volexity has been giving for years and years and years to organizations. And it’s often in an incident that we say, “Hey, your domain controller, for example, doesn’t need to be able to talk to the internet.” There’s obviously exceptions to the rules and everything, but usually those can be defined, especially with next-generation firewalls or modern firewalls — you can define what is actually needed, and allow them to do those things, and not allow them to do anything they’re not explicitly required [to do].

And that’s a model that is the least privileged, it’s like the least-access type model. That’s a little bit harder, depending on your organization, to enforce for users and workstations where they need to browse the web and do all this stuff. And that’s what content filters and certain restrictions are for. You know, unless you’re into, like, a DOD environment or something where it’s a lot more locked down. But that’s usually accepted in a lot of, like, commercial organizations.

And a server is where — an attacker, if they’re gonna install malware and do things — usually go for it, because that’s where the supply chain, that’s one of the big areas to get to it. Or those are the machines that are not at home, or requiring a VPN. They’re always on, you know, they don’t get rebooted frequently. That’s where malware gets installed a lot, because it’s something that they can count on, and it’s regular. Being able to prevent that, and limit what those can do — that model, if that had been put in place for organizations with SolarWinds — in this specific instance, it would have mitigated that threat.

Now, if I start thinking outside the box, and this attacker used DNS — but what if they had done command and control activity, and issued commands, and had done that all over DNS? So, the SolarWinds server talks to its local DNS server, your local DNS server goes out to the internet. If they had modified this malware and actually did all the command and control over DNS, instead of doing it over this connection, that paradigm and that shift would have been a lot more difficult to mitigate.

But that’s the type of issue and security item we need to think about. You could proactively try to address that, or just say, “Hey that’s a lower likelihood, and I’ll address it if that happens.” But by and large, it’s a best practice with regards to minimal access, specifically for servers connecting to the internet and different resources.

Joel: It’s funny talking about this, because it’s like the history of the security industry is the history of unreasonable requests.

I know that a lot of people are jumping up and down talking about, like, don’t let production talk directly to the internet. And if you worked at a bank you know for the last 20 years, that’s been the case, right. Like, highly-regulated industries, and people that have invested heavily on security, have always focused on doing these rather idiosyncratic things that don’t make a lot of sense — but made a lot of sense to people who either come from an incident-response or a deep security background.

You know, back in the 90s, I remember being involved in strenuous debates about why you need to encrypt traffic moving within your data center. And everyone thought it was the most asinine thing because it’s a private link. You’ve got MPLS, no one’s gonna listen to you — and then Snowden released his documents. And it became really obvious why you want to encrypt your data within your data center.

So, this is just another example where people have been giving best-practice advice, saying, “Hey, you need to make sure that random servers, random production systems, can’t just talk arbitrarily to the internet.” And the response to that has generally been well, that’s an unreasonable request, that takes a lot of work, I don’t know that we necessarily wanna do it. And, there was never a particularly great reason or piece of evidence to point you to say “well this is why”. So, this is why — why you wanna limit that access. And there’s probably a list of other things that are equally unreasonable requests that security people would ask you to do, and eventually they’re gonna have their “this is why” moment.

Steven: Something that Joel mentioned earlier, which I think is really important, is — a lot of organizations aren’t doing blocking and tackling. They don’t have two-factor authentication on the remote access to their network. They’re using weak passwords, they’re not patching. They don’t know where their assets even are, and their build process is not secured. They don’t even do code auditing or check-in their code. I mean there’s a lot of low-hanging fruit for most organizations. They haven’t even been able to kind of get into some of the basics.

But I think a big problem that a lot of organizations — whether that’s a government, commercial organization, or really anyone, whether they’re a small company or these massive companies with huge budgets — a problem that they’re facing is that if you had certain security data, you could immediately and very easily answer, “Did I have a problem?” One, did I run up vulnerable software? Because maybe, you patched. You know, I don’t know. Maybe I never ran, and I skipped a version. If you had all your DNS queries logged and the responses, you would say, did I get a C-name? Did I even call out to that command and control activity? There’re certain logs from the endpoints that SolarWinds has instrumented in these event-log data. If you had been capturing that data, you could answer that question.

Sonal: Most companies do capture that data, don’t they?

Steven: It depends. If you went into SMBs and mid-sized businesses, even some large businesses, I would say a lot of them aren’t actually logging or keeping DNS logs. And if they are keeping DNS data, it may not be query-and-response. And event logs — the vast majority of organizations don’t have a centralized and long-running retention policy for event logs. But even if they do, their data retention of how long they were keeping this data did not go back far enough.

They actually had data — they have data going back 30 days, they have data back 60 days, 90 days — so they’re finding out in December about a breach inside of activity that happened and then potentially initiated in May. And, “Oh, I kept all this great data, but I can only go back three months.” Three months from December, it’s September. And for a breach that happened in June or July, that’s, in some respects, useless. That’s a scary place to be in, to not know if you were compromised, or if you were when it started, or what happened, or where did they go, how did they pivot. It’s a missed opportunity, and probably a bit scary for some of these companies is that I was collecting all the right data, but I didn’t have it for long enough, so I don’t actually know.

Sonal: Wow.

Steven: We’re helping a lot of companies right now to see what resources they have. You know, we specialize in memory forensics. We’re acquiring memory from their SolarWinds server, acquiring disk artifacts, or full disk images, you know, any log sources. And we have some stuff that we can potentially go in and say “doesn’t look like it” or “definitely, yes you were.” You know, we see these items that clearly indicate that you got a second-stage breach, and you need to expand this out. But we can’t give anyone, if they’re on limited data, a confirmed clean bill of health.

Sonal: It’s a little bit like going to the doctor and having, like, maybe a continuous glucose monitor for the last year — but you only have the data for the last three weeks stored. And it’s sort of like, “Okay, here’s what’s happening. I’m getting sick, but I only have the three weeks.” It’s just, like, a really tough thing to figure out.

Advice for businesses and consumers

I wanna break this down by advice for big companies — like, large enterprises — advice for small and medium-sized businesses, and advice for consumers. So, let’s start with the big companies, because the best threat actors, they understand the reality of modern enterprise IT. What are pieces of advice — or mindsets, even — that you have to offer for how chief security officers, CEOs, leaders should be thinking about the implications of this for their business?

Joel: I mean, I’ve spent a lot of my career in big companies, and I think the thing to do right now is to think about strategy. Like, the tactics are great, and there’s gonna be a lot of people chasing a lot of actions over the next days, weeks, months. But I think the strategic view of how an organization wants to think about security — as we start to understand what happened, and how it happened — we’ll consistently see in some organizations that security either wasn’t funded, it wasn’t empowered, it didn’t have a remit to act. It may have been under assault. People often view security as being a cost center, as something that you know contributes to the lack of performance in a business. And that is an attitude that is still quite popular.

So, I would say that, like, it’s really gonna be about figuring out strategically, where does security sit, what’s the right amount to spend on it, how do you effectively empower it, and then how do you partner and build security into your business so that it’s something that helps enable it, versus something that holds it back.

Steven: Yeah. Generally, no one really thinks like security is not important. I don’t think we ever hear that. Now, action may speak louder than words sometimes. But I think a lot of people think about, “Oh, it’s an afterthought. I’m gonna add it later,” or “Oh, yeah, yeah, well, you know, we’ll do that one day.”

And I think, like, our main advice to a lot of these different organizations — whether it’s a startup or a midsize company, a company that’s growing really rapidly – is not necessarily that they need to come out of the gate and have to have every imaginable security product, they need to be auditing all their source code on day one, they need to have everything locked down, and the latest firewalls, and this filter and all these EDR products. But it’s like, think about that stuff. Are you doing the two-factor? Are you lazy, like, “Ah, I don’t need to put, you know, two-factor on my Salesforce account where all my most sensitive contacts and information is in my organization.” Or, “Ahh, I don’t really need to put it on email. It’s like, it’s easier if everyone can just log straight in.” Or, “I’m just gonna share this root, you know, Amazon key to get into AWS, because that’s just how our organization’s growing, and we’re not formal.” There’s things that people can do — best practices, actions that organizations can take — see what you can do now, see what you can do along the way, and put that on your radar, so you’re not in a position where you’re starting from scratch, or trying to investigate a breach, or figure out if you even had a breach. We all knew [what] we should have done, and we knew that two years ago. And we run into that, a lot.

Sonal: Don’t wait till later. And now advice for advice for consumers, like, just day-to-day people like family members, etc. What would your advice be for how to think about things like this?

Joel: We wrote a really excellent blog post last year called the “16 Things You Can Do to Protect Yourself,” and I would strongly recommend that people do all of those 16 things. It’s all really basic stuff, and it starts with two-factor authentication, patching your systems, and goes all the way down to how you want to think about securing your potential social media accounts, etc. So.

Steven: Yeah, we issued some guidance, and it’s a couple of intersections of prevention and detection, and then remediation, if you have an actual threat or concern.

From the prevention side, prevent unnecessary access from your servers — like, your SolarWind server, other devices — from talking to the internet. That’s a prevention mechanism. You know, monitor your assets, see where they’re logging in from, if you have that centralized logging or like a SIM, same thing. Make sure you’re capturing either from event logging or your endpoint security products or that the actual commands being run on the system are being logged. Because that can be pivotal and be critical to 1) detection — but even if you’re not actively monitoring it, you can go back and say, “Hey, what commands are running on this server that’s not consistent with what our system-admin or the typical activity would do.”

But take a look at your mail server, look at where your email is going, because that’s where the attackers, I believe — they’re way ahead of the game with regards to the things that they can do in Office 365 and Azure AD, where they are so familiar with the administrative commands and what to do from a sys-admin aspect. They’re able to do a bunch of things and hide in ways that people have never even thought about and encountered. And it’s not necessarily, like, they’re ghosts or they can’t be found, people just don’t know to even look for it.

And then, just from a general remediation perspective — once a device has been backdoored or compromised, it’s an untrusted system now. Don’t just, like, roll back to an earlier version, or, I’m just gonna upgrade to the new version. We say, hey — blow that whole system away. Start with a fresh, clean install. If you’re putting SolarWinds Orion back on it, download the newest version that’s not backdoored, and start everything from scratch.

Anything you used on that server, if your SolarWinds set up for the Orion had credentials, change all those passwords, and make sure those passwords aren’t similar to, like, old passwords that were used. You know, another thing, too, is — any sensitive API key integrations and things — like, we saw two-factor bypass to get into email by this threat actor. Because they had taken a secret key, and would generate cookies and skip into the email system while not actually being challenged for two factor.

You’ve got to think about the stuff that someone could steal if they’re in your network, related to this — but also that advice extends well beyond this threat actor and SolarWinds specifically.

Sonal: That’s great. I’ll include links to Volexity’s blog posts as well as the “16 Things That You Can Do to Secure Yourself” in the show notes. Bottom-line it for me — what’s your takeaway?

Joel: It’s consistent with what we’ve been saying for a while now. The hardest problem to solve is third-party risk, and this is probably the most significant third-party breach that we’ve seen in history. And so, I think it’s gonna take us months to really understand what happened, and probably years to fix it.

Sonal: Thank you so much, you guys, for joining this episode of 16 Minutes, which is a 3X 16 minutes.

Steven: Definitely, thanks for having me.

Joel: Yeah, thank you so much. And, Steven, it seems that we’re always catching up when the world is burning down.

• 

  Steven Adair
• 

  Joel de la Garza is an operating partner at a16z focused on information security related companies. Prior to joining the firm, he held top security roles at Box, Citigroup, and Deutsche Bank.
• 

  Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

  Follow Twitter

In one of our special “2x” episodes of 16 Minutes (32ish minutes;) — our show where we quickly cover the headlines and tech trends, offering analysis, frameworks, explainers, and more — we cover the algorithm that powers TikTok, the short video-sharing platform that grabbed massive marketshare in cultures and markets never experienced firsthand by the engineers and designers in China, beating out other apps in the United States. Now, with talk of U.S. ownership/partnership for TikTok, what happens if the algorithm isn’t included in the deal? And what can we learn from the “creativity network effects” flywheel of TikTok; for “algorithm friendly” product design; and more broadly, about the future of video?

The news: Given the U.S. government calling for TikTok’s business to be sold to U.S. owners last month, and several bidders coming in since, the latest news was that Oracle Corporation and Bytedance are hammering out an agreement for the former to be TikTok’s “trusted tech partner” in the U.S. This could include (as reported by Axios) their exclusive ability to oversee all tech operations for TikTok in the U.S., including access and control of U.S. user data; ability to review source code and all updates to software for security vulnerabilities; and separate boards and entities for ensuring compliance with CFIUS/ U.S. policies (and for allowing ownership stakes for Oracle, with Walmart). The deal hasn’t been approved yet [as of September 18, 2020].

The episode: But since this show is focused on where we are on the long arc of innovation, and what’s hype/ what’s real when it comes to tech trends & the news, where does the source code (and more specifically, the “For You Page” algorithm) — which may or may not be included in the deal due to China’s revised export controls — come in? Yet it’s not just about if TikTok is really TikTok without it, or whether “the algorithm” and machine learning training data can be recreated… the real question is: How does the “creativity network effects” flywheel work between video creation and distribution — from origination to mutation to dissemination? It boils down to the idea of “algorithm friendly design”, observes Eugene Wei, who has written a series of deep dives on TikTok, and formerly led product at Hulu, Flipboard, and video at Oculus, among other things. So what does TikTok, regardless of deal outcome, suggest about the future of product development, and more broadly, the future of video? All this and more in this 2x+ long explainer episode of 16 Minutes.

Show Notes

• An overview of how TikTok’s algorithm came about [2:11] and the creative tools that the app offers users [5:23]
• How TikTok encourages reusing and remixing [9:16] and serves up content based on the algorithm [14:16]
• TikTok’s intensive tracking of user behavior [18:07] and how the algorithm sorts users into communities [22:30]
• Issues around the possible acquisition of TikTok [27:46]
• TikTok’s effect on commerce and video [30:00], as well as the merging of content internationally [34:50]

Transcript

Sonal: Hi, everyone. Welcome to “16 Minutes.” I’m Sonal, your host, and this is our show where we tease apart what’s hype and what’s real when it comes to the headlines, the tech trends, and where are on the long arc of innovation. And so, this episode is all about the short video-sharing platform TikTok (which has been in the news a lot lately), but is also about the future of entertainment and especially video. We also cover “creativity network effects” from creation to distribution, the concept of “algorithm friendly” product design, and much more.

For those who are new to this show, I do one of these deep-dive, kind of “2x” explainer episodes — so about 32ish minutes) — every so often, where we talk about what’s in the news, but really dig into — with the top experts — the key underlying concepts. And our expert today is Eugene Wei, who has written a series of deep dives about TikTok, and formerly led product at Hulu, Flipboard, and video at Oculus, among other things. (As a reminder, none of the following should be taken as investment advice; for more important information, please see a16z.com/disclosures.)

And for the quick news context before we go into the discussion:

• TikTok has obviously been in the headlines with the administration calling for its sale and majority ownership of it in the U.S. last month, with multiple companies bidding since;
• The latest news, as reported by Axios, is that Oracle and Bytedance are hammering out an agreement for the former to access and control U.S. user data; to have the ability to review source code and all updates to software for security vulnerabilities; and have independent boards for compliance;
• But all of this is yet to be cleared by both governments.

So, our focus in this episode will be around the evergreen and key question of where the algorithm (as if it were a single thing!) does and doesn’t come in — given talk of removing it from the equation. And more specifically, the “For You Page” algorithm, which, Eugene, you wrote about recently as quote, “the most important piece of technology” that Bytedance introduced to Tiktok, and you also called it “the hardest part,” which allowed a team of people who’ve mostly never left China to crack the cultural code and grab massive market share in places they’ve never experienced firsthand. So, what do YOU make of the news that this sale or partnership or whatever it ends up technically being, may or may not include this algorithm?

Eugene: Yeah, I think in a lot of talk about TikTok’s algorithm (and I’m partially responsible), the dialogue’s gotten a little bit breathless, around the algorithm — it’s become like the magical MacGuffin in a film; the, you know, suitcase of whatever in “Pulp Fiction” (or something like that).

And, while I do think the algorithm is important, I actually think that people may be overstating just like the power of the algorithm in isolation, whether it comes along in a deal or not. If you ask machine learning researchers around the world, if they think ByteDance has some algorithm that nobody has, I doubt they would agree; the algorithm is based off of very conventional research, and conventional thinking in terms of recommendations algorithms. What matters is actually the combination of the algorithm itself, and then the training data that you can train it on — and it’s the combination of the two that’s super powerful.

But, what makes TikTok different from other spaces (like visual AI or text AI), is that there isn’t a large corpus of just publicly available training data. And so the magic of TikTok in a way is that it’s a closed-loop ecosystem: It’s an app that encourages its users to create the training data that it then trains its algorithm on. And that’s I think, the magic.

Sonal: Can you quickly actually just walk us through the history of how TikTok actually did get that training data and then combine the algorithm to create this phenomenon where it was able to run circles around U.S. video apps, from YouTube, to Facebook, to Instagram, to Snapchat — How did they do that? Because anyone could have theoretically, you know gathered training data and come up with a different algorithm; like there’s something specific here.

Overview of TikTok’s algorithm

Eugene: Yeah. Well, it’s ironic because it starts with the app Musical.ly, in many ways.

Musical.ly was a video app created by Alex and Louis, who had worked in the U.S., but were in China, and had pivoted from a short video education app. And, they launched it in both China and the U.S. — and it actually became more successful in the U.S., especially among American teenage girls, who used it to do lip sync and dance videos — then ByteDance cloned Musical.ly essentially, in China, in an app called Douyin. The irony of that is actually that the clone of Musical.ly ended up launching in a larger market, and becoming a larger app with a larger user base. And so eventually, they bought Musical.ly after its growth had stalled out in the U.S. And that’s when they rebranded Musical.ly into TikTok.

So it’s this weird you know “multi-hop” mutation of the app that like <chuckles> — built in China; did well in the U.S.; got copied in China; and then China bought the U.S. version — it just kept hopping back and forth across the ocean.

Sonal: Well now the hop is kind of funny because it could go the other direction <yeah>, where part of it could be divested to a sale in the U.S.!

Eugene: Yeah… it just keeps going back and forth.

But, all of that wouldn’t have mattered if nobody was making videos on the app, right; they actually had to build an app that made it possible for people to create a new type of video.

Unique creation tools

Sonal: Could you break down a little bit more into the tools? You come at this from the vantage point of someone both in *tech, and who’s also been to *film school, and is a huge lover of multimedia. What specifically — let’s talk a little bit more about what makes the tools — because frankly, there’s a lot of apps in the U.S. (like YouTube and others) who easily have the capabilities of putting these tools together.

Now they didn’t — so that’s part of the point — but what specifically about these tools or the combination about them is really part of this flywheel?

Eugene: Yeah. That’s where the app is a little bit underrated in terms of its creation tools. It has a really great set of camera tools; editing functions; filters that take certain high-production film techniques, and make them really accessible to a broad audience. Even licensing the music tracks was a huge thing for Musical.ly to do: Previously, if you wanted to lip sync to a pop song, you had to get like a pirated copy (or just do something that might get pulled down for copyright and trademark violations). Them signing the deals with the music labels now allow teenagers to lip-sync to the actual version of the song that they wanted to lip sync to.

Sonal: That’s a great example of a tool that really makes something easy and fast, that was previously hard.

Eugene: It’s two things; one is, the creation tools are really taking features and functions that traditionally you would have to use like the Adobe Creative Suite to do, on your laptop — and making it possible to do a lot of that just with your phone. That’s a huge thing because first of all, a lot of people can’t afford Adobe Suite tools, and the learning curve on them is significant; if you didn’t go to film school, you don’t know how to use After Effects. But TikTok essentially integrates those into kind of their camera suite.

The second thing I think — and this is less about the tools — there are network effects on the creativity side, when it comes to TikTok, and that’s really underrated.

In your podcast library, you probably have a ton of episodes that are all about all different types of network effects; the important thing to think about when it comes to this example though is just that: Does every additional creator on TikTok, make the rest of the community more creative? That’s what I mean by creativity network effects. And I actually think it’s very rare to find this form of network effect in the wild, but TikTok has achieved it, a couple ways:

…So the hardest thing for any creator, on any app, is to just think about what to create. You know, if you are presented with a blank canvas or the blank page as a writer, can you come up with something from scratch. And the truth is, most people can’t originate ideas.

But TikTok — because of the distribution, because of their discover page making what’s trending very salient — essentially allows you to just remix someone else’s idea. Most TikToks that people make, are actually just riffs on someone else’s idea. And so they solve that sort of blank page problem for you. You can go on TikTok and find a whole bunch of ideas, from other people.

…The second thing is they actually structurally make it possible for you to physically riff off of the other person’s idea. So, you could do —

Sonal: Oh you’re talking about Duets, yeah.

Eugene: — a duet; yeah you could do a duet with someone where just like one half of the video with someone else.

You can easily grab a component of their video to reuse in your own — like maybe you just like the music track, and the music track is the meme that you want to make; now you can just grab it, reuse it. And sometimes people upload original audio; so someone just records a TikTok video from scratch, you can even just use their audio, in your own TikTok.

…And, the last thing is just really, I think there is a shared inspiration in the community — they make sure that if someone comes up with an inspired idea, it’s distributed really broadly. And then the sort of ethos of TikTok is that you pay it forward, everybody can borrow somebody else’s ideas.

Remixing and reusing content

Sonal: So, it’s really interesting because you in your original post described,“TikTok is such a fertile source for meme origination, mutation, and dissemination”.

So we’ve talked about the origination, which is like the creative tool suite. You’re now talking about the mutation, which is this remix, taking bits and pieces — I feel like a broken record because I often talk about “combinatorial innovation”, which is such a buzzword — but it is sort of this idea of remixing bits and pieces, Lego blocks, composability in software; there’s many ways to describe this phenomenon.

But specifically on the mutation side, it makes it very easy for people to be creators without having to be “creators”. What do you make of challenge culture within that too, and hashtags, and some of the other specifics within TikTok, that kind of make the mutation work? Because again, remix culture is nothing new; in fact, when I think of the early web, the story of it is remix culture. So like what do you think specifically about TikTok really advanced the mutation… wheel?

Eugene: Yeah. I think that’s where the algorithm actually really comes into play — because the algorithm determines kind of who sees what. So, there’s a way in which you are incentivized to participate in certain challenges because you know the algorithm happens to be amplifying that particular meme and trend a lot right now.

If you didn’t have the algorithm, and things had to organically find an audience, that whole challenge culture thing would work so slowly that it might not actually achieve critical mass. In a way, what TikTok is, is a mix of a free market — but also a managed economy.

Sonal: Ooh, interesting.

Eugene: So on the Discover page (which is a tab that you can go to), they will post what are the challenges that they’re featuring at the top: What is the hashtag; what is the you know musical track that fits with it; and what are people doing for that challenge. And you know as a creator then, that if you make something on that challenge, you have a chance to hit the top of the Discover page because it’s being featured.

So that’s the managed economy part of it, where they actually can coordinate the entire community, and create common knowledge about what is going to be promoted. And it’s the same with hashtags, right; the hashtags that you can search on, you can see how many views each hashtag is getting right now, and try to attach yourself to the ones that have the highest velocity and momentum.

Sonal: Right and as a quick point of contrast for those who are not as… as, in TikTok <chuckles>; in contrast, when you think about most other social networks and the trending hashtags, you actually don’t know which is more– the weighting of them at all, they could be arbitrary for all you care; <Right> it could be five people trending, it could be whatever.

And then similarly, one of the biggest complaints people have had about YouTube is that you CAN go viral, but it’s very rare, and it’s very loaded towards very established people, as a mature established platform, because you’re essentially “gaming the algorithm.” And so what you’re kind of saying in a weird way here as you can game but not game the algorithm, <yeah> on TikTok.

Eugene: And it does feel meritocratic in that way. You’ll sometimes click into a profile, of a creator who’s made a viral video — and you’ll see that all their other videos actually have very low view counts. They’ve sort of removed that old money effect that I describe in other social networks, where the creators who’ve been there the longest, have such an advantage over new creators.

Sonal: Right; they’ve accrued the most “status” in that network…

Eugene: …Exactly, exactly. So if you even see like the Meteor/Meatier pun video this week — which is about the extinction of the dinosaurs — that one was great, because she was kind of a newish creator who finally just had that first big hit.

@lizemopetey

is this too soon…? IB climaxximus on twitter #fyp #dinosaur ThatsHot #DinnerWithMe #MorningCheer

♬ original sound – Eliza Petersen

<Sonal: Ah that’s great> And that also helps on the viewer side, right — because you’re not getting decreasing economies of scale, where the same creators videos keep getting shown to you, even if they’re no longer any good. You are always being shown stuff that they have determined, has entertained some test audience, at some you know part of the network.

Sonal: It’s almost like evolution; it’s constantly testing for fitness <right> of this creator, essentially in this, in this model.

Eugene: Right! We know from evolutionary theory that the stronger the fitness function or the selection pressure, the better the output on the other side.

And I view TikTok as an “assisted evolution” ecosystem: It’s not purely leaving everything up to chance — they do put their finger on the scale sometimes in terms of hey, we have a corporate partner that wants to do this challenge; we’re going to feature it, and that’s going to give it more prominence — but for the most part, no matter how popular you are as a creator, they’re gonna let your video sink or float based on how it does with that first test audience they show it to.

Sonal: So when you talk about assisted evolution, it’s like a combination of this managed economy and free market dynamic, which is fabulous. <yeah> Okay.

So, so far then these are all the kind of features that now we’re kind of wrapping up on this idea of mutation. So TikTok being the most fertile source for origination with the creative tools, and, those allow some more of these creative network effects. The mutation, which allows this interaction of the community, the discovery; the fitness of creators — so you’re always getting fresh, and not only going with only the mature creators — and other kind of dynamics to play in this assisted evolution as you describe it.

How TikTok serves up videos

So now let’s talk about this “fertile source” for dissemination — and by the way, I don’t mean to cut these apart as if they’re three discrete things; they’re obviously on a continuum, and interact — but let’s talk about dissemination and really, distribution.

Eugene: Yeah. So, the algorithm essentially sits at the center of all this; the algorithm is going to determine who gets shown what videos. And creators are only going to go typically, to a network where they feel like they have a chance to get disproportionate distribution of their content.

And, the way that TikTok has sort of like short-circuited that process and accelerated it, is by using an algorithm rather than a social graph, as the primary axis of distribution.

Sonal: Say a little bit more about what that means just for our listeners who are not in the weeds of, social networks.

Eugene: Right. So in a typical social network, like Facebook, or Twitter, or Instagram, you start posting content, and then you try to acquire followers — and this builds out kind of a social graph, right; it’s an interconnected web of people. And based on who chooses to follow you, you will get distribution of your content to them. And then eventually if the network gets really big, they’ll put some algorithmic feed into place, where not everything you create will be shown to the people that follow you.

I always think of this as the very traditional path of social graphs, where the follower graph kind of determines the pathways through which content travels.

Sonal: Which is then very path dependent, shaping the future of that social network.

Eugene: Exactly. And so, if you don’t build up enough of a following, eventually your content gets no distribution; you’ll churn out of the network, or maybe just become a viewer, where you only look at other people’s work.

TikTok doesn’t go through that process at all. They have the ability for you to follow creators, but, that content is put into a secondary tab, the Following tab — which gets like just a fraction of the traffic that the FYP tab gets.

Sonal: Which is the For You Page.

Eugene: The For You Page. Essentially, they use the algorithm to determine what you see. And that just allows you to see content from people that you don’t follow, that you would enjoy otherwise. And I call this just you know TikTok basically fast-forwarding to the interest graph and bypassing the social graph.

Traditionally, our large social networks in the West have consistently used a social graph to approximate an interest graph. But that gets them into problems.

Sonal: Yeah… In fact, if you look at the history of original recommender algorithms, I actually met the guy who got the original patent on he used to work at Xerox PARC. And one of the things that’s fascinating about that is that he had this really cutting-edge insight [at the time] that one of the ways to recommend things is to look at your friends and find things that you like. But that’s not always true. Like, your friends’ interests do not actually capture your interest. Like, I’m your friend, and I love your views on film and you’re really into movies and books; I have those interests in common with you — but you’re also really into sports, and I have no interest in sports. And so if you were suddenly tweeting a bunch of sports things, I wouldn’t be interested in following that segment of your timeline.

Eugene: Right, so we’ve seen this happen again and again in other social networks: On Facebook, they pivoted from, hey here’s photos from your friends, to hey here’s someone sharing like a political news story. And it’s the same on Twitter where you might follow someone who has a lot of interesting thoughts on something that you care about. But then, yes, they suddenly start posting about their favorite home sports team, or, something that you don’t care about — and then you’re stuck in this bind, because the entire feed, and the entire graph, is built off of that social following. And you start to get a higher noise to signal ratio in your feed. And that can lead to churning, or losing interest in that.

So TikTok is like you know what, we’re not focused on that at all: We just consistently want to know what’s entertaining you right now. And we’re going to keep showing you more of it.

Tracking user behavior

Sonal: I’m just gonna read something from your post that’s super relevant, because you talk about how they notice everything. And if you like a video featuring video game captures, “that is noted”. If you like videos featuring puppies, “that is noted”. Like, “it is known”, it is noted, it is noted. So they notice everything basically, and they do all the work, so you don’t have to explicitly tell the algorithm by who you’re following… it just decides for you and serves things up to you.

Eugene: The thing that’s really interesting, is that they epitomize an idea that I first read about in James Scott’s Seeing like a State.

James Scott writes a lot about hey, you know a lot of modern governance and everything was built around this idea of, we have to make certain phenomena more legible in order for us to take actions on them. For example, if you want to tax people, if you want to conscript people, you need to actually know like how many people live in your country, what pieces of land do they operate; and so, there came about this idea of just classifying and structuring society in a way that made those units of measurement more legible, so that you could do things like tax people fairly. And we live in such a world where that’s taken for granted now that we almost don’t think about it, but if you think about a previous era, when people didn’t even have last names, it was just really hard to track your citizenry.

I think about TikTok as an app that epitomizes the idea of “seeing like an algorithm” — where if the algorithm is going to be one of the key functions of your app, how do you design an app that allows the algorithm to see what it needs to see?

So, the ByteDance example: They have a huge operations team that when videos are made, are tagging videos with features and attributes — so this video has a kitten in it, this video has a lion in it, this video has soldiers doing workouts in it. All those classifications actually really matter because visual AI hasn’t reached a point where you can determine exactly what the video is about. But because ByteDance invests so much in this, when they serve a video to you in TikTok, the algorithm can already see a lot of what’s in the video, it knows what the video is about.

Next, if you look at the design of the app, what’s striking about TikTok is it only shows you one video, full screen, at a time. And whether it’s by design or accident, this is very very different from social media apps, where there are many items on the screen at one time. So with a Facebook or Twitter, if they show you like four stories on your phone screen at a time and you’re just rapidly scrolling past it, the algorithm has a hard time seeing what you feel; like, what are you even looking at on the screen?

TikTok is different: They show you one video, one video only. And from the moment that video is on the screen, they’re looking at everything you do. And they can attribute all of that to being a clue as to your sentiment on that video. If you flip past that video, before it even finishes, that can be a negative signal. If you instead let the video loop four times, then you share it, then you heart it, then you go and follow the creator, or then you go and look at the musical track — those are all signals of interest.

And so in that way, their feedback loop is super efficient and tightly closed. And that is, I think, a form of design that I refer to as “algorithm friendly design”. You know traditionally, all of the design principles that have guided the Valley for a long time are about minimizing user friction; in this case, they’re actually introducing a bit of friction, right.

It would be faster if they showed me multiple thumbnails on the screen, for me to just scan through a bunch and flip through them; they’re intentionally slowing me down, and showing me one thing at a time. But in doing so they get much cleaner feedback about my sentiment — and that means that the training of the algorithm happens more quickly.

Sonal: Oh my god, what a great explanation. So just to quickly sum up, this idea of “seeing like an algorithm” is critical. And what you really added to this as well — besides that great phrase <chuckles> — is, the fact that the product is designed to support this ability to essentially isolate the variables, in that feedback loop of what you’re studying and what you’re noticing, so that you feed it back to your users.

Sorting users into communities

That explains then the context that we need to know to kind of understand how the algorithm works, and what it is. So now let’s cover the third question of dissemination — and now how does that play into this whole… flywheel of these creator network effects, and then now you have distribution.

Eugene: Yeah. So, the problem in the modern age is not that we don’t have enough content… it’s that can that content find its audience. And because TikTok has such a nice closed feedback loop — its algorithm can see what each viewer is interested in, and it can see what each video is about — it can also see how an initial test audience reacts to a video.

It has all the components it needs to match the right video to the right viewer. And that’s the distribution part — not built on a social graph, built on an algorithm that’s just really efficient at matching content, to people who will enjoy that content. And that’s why I referred to it as “The Sorting Hat” from Harry Potter; you know more about Harry Potter than I do.

Sonal: <laughs> I do!

Eugene: Yeah, it’s a little mysterious how the Sorting Hat works. But it did seem to pick people with the right disposition to be a Hufflepuff, or a Gryffindor, or a Slytherin.

You know I’m interested in really weird postmodern memes on TikTok, and it consistently serves me some really bizarre things <chuckles>; it feels like magic to me. But I know that it’s very mundane if you break it down how it works.

Sonal: So, just to just to ground the significance of your analogy of the Sorting Hat — Imagine a world of the countless thousands, millions, billions of users out there. And then you have… this ability to essentially identify people who have like-minded kind of interests — again going back to the concept of interest graph — and sorting them into quote-“houses” of shared interest. Because in Harry Potter, the analogy is not just that these people are alike or anything, but that they have shared interests, and personality traits, or things that they like, or whatever it is.

You know one of the interesting things about the internet, is people often talk about how it breaks down geographical barriers… going back to this idea of the Sorting Hat, the significance of this ability to distribute and sort people into houses, and communities, is really significant.

Eugene: The thing that an algorithmic sorting allows you to do is to just scale that sorting function… infinitely. You could have editors at a magazine trying to determine what its readership is interested in, but, it will never be able to keep up with the just sheer infinite variety of its audience. You could have Reddit, which kind of sorts people into subreddits; but you still have to go and find the subreddit yourself and join.

TikTok just allows this to happen organically, without you really having to do much that feels like work. They don’t necessarily force you through a long profiling step; you just jump in and start watching these funny videos. It’s relatively low cost; if you see a bad video or one that bores you, you just swipe past it, and immediately have a new one playing. And as that’s happening, the app is learning about your tastes.

The other thing is people’s tastes change, over time. And so as your tastes evolve, the TikTok algorithm quickly can detect that like oh okay, this week you’re into Draco fan fiction. We’re gonna show you some more of that, because we happen to have plenty of that right now–

Sonal: <laughs> Which you are!

Eugene: –Yeah yeah; and I’m sure by next week, I’m going to be on to something else. <Right> So it sort of is just closely hewing to your taste profile.

You know, Alex and Louis (who founded Musical.ly), I mean they did work in the U.S.; so it’s not like they didn’t know anything about American culture. But, the fact is that no matter how many people you have working at your company, there’s no way — if you reach hundreds of millions or even billions of users — that you can personalize, manually for all of those users. And, the algorithm here essentially says that you can scale to serve an audience of ANY size, in ANY country. And that’s really powerful.

Sonal: So just as you made the observation earlier that the creators can evolve on this platform, and that the system evolves in identifying them and their skills as they do, so does it work for the consumers who are evolving, which is super powerful.

I love what you said about the subreddits, too, because it’s not just the friction — actually, when you go into any kind of online community, you have to learn these norms. And here, you’re kind of immersed in a community; but, it’s actually not social at all, at the end of the day. Like TikTok, ironically, is not a social network, technically, then. <Right!> How do you kind of define it in your taxonomy of social networks?

Eugene: I call it an entertainment network, where its primary purpose is to match these entertaining videos from creators, to the audience that would enjoy them — that’s its primary purpose. And you can obviously leave comments with creators… And a lot of creators will accept challenges, from their viewers (you can ask someone to make a video of a particular type, and sometimes in a video, they will say, “Hey, this is in response to user X, Y, Z”) —

But you’re right, that the dominant mode of TikTok is not as a social graph. And that’s probably by design, and allows them to avoid the negative economies of scale that come from a social graph, that reaches a really large size.

Issues around acquisition

Sonal: Okay. Now let’s bring it back to the news and the trends; so this show is about covering the long arc of tech trends — we’ve talked about the evolution of recommender systems, the social networks, we’ll talk about video in a second — you’ve started to tease apart what’s hype, what’s real (including some of the hype you yourself may have put out about the importance of the algorithm;) —

To close the loop on bringing it back to the news, where do you stand on this idea, if in the final agreement — and again, who knows what’s going to happen ‘cause this changes every day — the algorithm is or isn’t part of it? Cuz China just updated their export controls to be able to refute the deal if they don’t want it to be in there, the source code. How much of a difference do you think it makes? Do you think if they were to back engineer an algorithm that functions similarly, that noticed everything, given the current product design — do you think they could conceivably still recreate that sort of wheel, given that there is already this critical mass of users on TikTok?

Eugene: Well, earlier, I talked about how I think people are maybe overrating the algorithm in terms of just like you know how unique the algorithm is itself. But: It is certainly true that if you purchase TikTok and it didn’t come with the algorithm, it would take you some amount of time — even if you had all the user data, video metadata, all of that — to sort of rebuild, and retrain, an algorithm of your own.

There’s always a risk with a social network that in that interim period (maybe it takes you months, maybe it takes you a year), that people would find that the app wasn’t as responsive… to their interests anymore, and that they might churn off of it. So, certainly you would rather have access to the full closed loop that allows that information to be fed back cleanly into the algorithm.

The algorithm’s already been trained; the hardest part often with a lot of these algorithms is getting that training data set, and they already have just a massive training data set of these videos with I don’t know, a gazillion hours of view time. You have a lot of users whose tastes are — have already been profiled.

So, yeah. I would say that it is possible to rebuild an algorithm. I think with the right tech companies, you have a lot of the talent here in the U.S. that can do that. But, that process takes time, and that’s risky.

Broader effects of TikTok’s approach

Sonal: Okay, so now I’m going to ask you just two last quick questions on sort of the long arc of tech trends, and then one practical question before we switch to that.

As someone who thinks a lot about product, and multimedia, and you know has worked on designing– you’ve actually actively designed many of these things in production, do you have any advice, or what are the implications, of all this — besides the fact that this phenomenon could occur, penetrate into mass market — what do you think about how this affects your thinking for finding product-market fit, or designing products in this… kind of era?

Eugene: Yeah. You know, I think a lot of people have said wow there hasn’t been any big new social network in recent years other than Snapchat, that have come up to challenge Facebook, Instagram, Twitter, those giants. So I think one big learning from TikTok is, hey, there’s an alternative approach that might work — which is to just cut straight to the interest graph.

And… that the way to do that would be to figure out, can you design an experience, a user experience, that allows a machine learning algorithm to get access to a unique set of training data. And I think it is probably possible in other fields and disciplines. I do think it takes a new approach to design, which is this “algorithm-friendly design”.

Sonal: Yeah. “Seeing like an algorithm”.

Eugene: Yeah, exactly. You’re like hey, this algorithm isn’t sitting in this design meeting with us right now; but it’s really important that when we’re thinking about what does the UI look like, what are the feedback loops, that we’re capturing the right data for the algorithm to be able to SEE, and do its work.

So I think that is a novel new sort of design- and product-development paradigm, which TikTok has created. (And you know really ByteDance even used that to develop their first trendy hot news app in China called Toutiao.)

Sonal: Okay. So then now arcing back up a bit to broad trends, how do you view this in the long arc of innovation when it comes to video, and the future of video? Because one of the recurring themes of your post — which it was kind of a recurring motif — is, we really haven’t figured out video; we’re actually still at the beginning of video; there’s a lot more to be done in video; it’s shocking to me how little people are doing video well. What are your high-level takeaways on that front when it comes to that tech trend and evolution?

Eugene: Two big takeaways. One is, that I think we consistently underrate the degree to which people respond more broadly to video than they do to, for example, text. You know the number of people who are going to read books, all the time, is just a fraction of the number of people who enjoy watching video.

And so, that really matters at scale. When you’re talking about reaching a broader audience. I don’t think we have a medium that can challenge video, in the world. I think the evidence is overwhelming. The second thing is…

Sonal: I mean, I would give you a little bit on audio <Eugene laughs> but we don’t have to go off on that side tangent. Let’s just stick to video <right> Keep going! The second thing.

Eugene: The second thing… is that in order for video to scale as a medium, you do have to do some work to overcome some of the challenges inherent to video. Video is traditionally a little bit harder to scan for conceptual information; you know, it’s harder to understand what’s in a video. Even if you’re watching a video, if someone sends you a video, sometimes people are like, I wish you would just send me the transcript so I can just scan through it really quickly. You know scanning video is even hard.

So, TikTok fortunately, the video is all really short. And they allow additional layers of metadata; you can bring text into the video, really easily. And so video overall as a medium, is a richer medium on TikTok. If you can bring that all to bear, then I think video becomes more relevant in other fields — like, education, or you know if you want to pick a place to go on vacation, or you want to pick a restaurant to go eat at.

Sonal: Yeah. Our partner Connie Chan’s actually argued a lot about the power of every commerce will become video, and every video will become commerce, and sort of the intersection of the two.

Eugene: Right… video is really just the bed for a whole bunch of other information to be laid on top of it.

Video is just such a high bandwidth medium. I think we haven’t really taken advantage of that full level of bandwidth in the past. We know that humans are super attuned to body language, to reading another person’s face; you know, one of the downsides of trying to read body language over Zoom, you may have like 15 people in a Zoom, each is just a small thumbnail; you can’t really see anything, other than a blurry version of their face. There is something that is lost when you lower the bandwidth. And video brings that back, and video gets higher fidelity every day. And, you know something like TikTok now is just making more use of that full bandwidth.

Sharing content internationally

Sonal: Great. So then the last question: What do you make of this larger phenomenon, given that the whole point of your post is about how this is the first time a social network from another place has really cracked into a different market. (And we haven’t even talked about India, and Middle East, but it’s also cracked into other markets, not just the U.S.)

The thing that fascinated me about your post, is this idea that there could be this internet layer that crosses regions and cultures. And you share an anecdote at the end of your post where the engineers that you — the office that you visited, they had like all these Hindi lyrics and Bollywood lip synching going on, and not a single person in the office even knew what they were seeing, or could even read Hindi. That is kind of amazing…

Eugene: Right. And that’s the one powerful thing about video; a lot of it doesn’t require you to understand the language. In fact you know a dance video, a little skit, even if they’re speaking in it, often you can just interpret based on what’s happening on screen <yeah> what they’re talking about.

That language is international. In a way, it’s more international language than even text. You know a lot of people in America still can’t read a lick of Chinese <mhm>, and a lot of people in China can’t really read English. But when it comes to video, and you show somebody a video on your phone, everybody can understand you know, oh this is a cute baby video, or this is an animal doing something funny. Netflix, for example right, is trying to figure out, hey which shows that we make in one market could carry over to other markets; if we can, we prefer that because it makes our content spend more efficient.

Sonal: All right. So, Eugene, bottom-line it for me. A lot to say, but on this explainer/news commentary episode on to-algorithm or not to see-like-an-algorithm, what is your takeaway on the news, bottom line it for me.

Eugene: Look, I don’t know what’s going to happen with this deal… regardless of that, I think TikTok’s impact will last, in that it provides a model, for how in an age of you know increased use of machine learning algorithms, you might build a new sort of network — that’s really built around algorithmic recommendations, and that shortcuts you to building out the interest graph.

Which ultimately, is probably one of the most valuable graphs in the world. If you think about how social networks make money — trying to determine which ads are relevant to serve to you; on the other side, the advertisers want their ads to reach the right audience — that’s ALL interest graph; that’s not really social graph. And so TikTok came along at a time when everybody was like, well, we’re stuck with these social networks. And they kind of snuck up on everybody from the side. And that’s a remarkable story.

Sonal: Thank you so much for joining this segment of “16 Minutes”, Eugene.

Eugene: Thanks for having me.

image: Eliza Petersen

• 

  Eugene Wei
• 

  Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

  Follow Twitter

In this special “2x” explainer episode of 16 Minutes — where we talk about what’s in the news, and where we are on the long arc of various tech trends — we cover all the buzz around GPT-3, the pre-trained machine learning model that’s optimized to do a variety of natural-language processing tasks. The paper about GPT-3 was released in late May, but OpenAI (the AI “research and deployment” company behind it) only recently released private access to its API or application programming interface, which includes some of the technical achievements behind GPT-3 as well as other models.

It’s a commercial product, built on research; so what does this mean for both startups AND incumbents… and the future of “AI as a service”? And given that we’re seeing all kinds of (cherrypicked!) examples of output from OpenAI’s beta API being shared — from articles and press releases and screenplays and Shakespearean poetry to business advice to “ask me anything” search and even designing webpages and plug-ins that turn words into code and even does some arithmetic too — how do we know how good it really is or isn’t? And when we things like founding principles for a new religion or other experiments that are being shared virally (like “TikTok videos for nerds“), how do we know the difference between “looks like” a toy and “is” a toy (especially given that many innovations may start out so)?

And finally, where are we, really, in terms of natural language processing and progress towards artificial general intelligence? Is it intelligent, does that matter, and how do we know (if not with a Turing Test)? Finally, what are the broader questions, considerations, and implications for jobs and more? Frank Chen (who’s shared a primer on AI/machine learning/deep learning as well as resources for getting started in building products with AI inside and more) explains what “it” actually is and isn’t; where it fits in the taxonomy of neural networks, deep learning approaches, and more in conversation with host Sonal Chokshi. And the two help tease apart what’s hype/ what’s real here… as is the theme of this show.

image source: Gwern.net 

Show Notes

• What is GPT-3, how do developers access it [3:56], and how is it different from other machine learning tools? [6:52]
• Discussion of how to categorize GPT-3, how it learns [9:57], and where it fits into the AI big picture [13:43]
• Real-world applications and scalability [16:20]
• How existing technology companies may respond to tools like GPT-3 and further iterations [19:16]
• Discussion of how people may work with tools like GPT-3 in the future, and how it could transform the workplace [20:54]
• Ethical concerns around stereotyping and racism in AI [24:34]
• The need for a new Turing test for AI [26:47] and predictions for the future [30:18]

Transcript

Sonal: Hi, everyone. Welcome to this week’s episode of “16 Minutes.” I’m Sonal, your host, and this is our show where we talk about the headlines, what’s in the news, and where we are on the long arc of tech trends. We’re back from our holiday break, and so this week we’re covering all the recent and ongoing buzz around the topic of GPT-3, the natural language processing-based text predictor from the San Francisco research and development company, OpenAI.

They actually released their paper on GPT-3 in late May, but only released their broader commercial API a couple of weeks ago. So, we’re seeing a lot of excitement and activity around that, in particular, although it’s all being called GPT-3. So, we’re going to do one of our explainer episodes. It’s a 2x explainer episode going into what it really is, how it works, why it matters, and broader implications and questions while teasing apart what’s hype, what’s real, as is the premise of the show. But before I introduce our expert, let me just quickly summarize some of the highlights.

So, while GPT-3 is technically a text predictor, that actually reduces what’s possible because, of course, words and software are simply the encoding of human thought — to borrow a phrase from Chris Dixon — which means a lot more things are possible. So we’re seeing, and note these are all cherry-picked examples — believable forum posts, comments, press releases, poetry, screenplays, articles, someone even wrote an entire article headlined “OpenAI’s GPT-3 may be the biggest thing since Bitcoin,” and then revealed midway that he didn’t actually write the article, but that GPT-3 did.

We’re also seeing strategy documents, like for business CEOs and advice written entirely in GPT-3. And not just words, but we’re seeing people design, using words, to write code for designing websites and other designs. Someone even built a Figma plugin — again, all of it showing the transmutability of thoughts to words, to code, to design, and so on. And then someone made a search engine that can return answers and URLs in response to “ask me anything,” which as anyone who’s been in the NLP space knows. I was at PARC when we spun off Powerset, back in the day, and that’s always been sort of a holy grail of question-answering, which you know all about too having worked in this world, Frank.

And now, let me introduce you — our expert in this episode. Frank Chen has written a lot about AI, including a primer on AI deep learning, and machine learning, a pulse check on AI, what’s working, what’s not, a microsite with resources for how to get started practically and do something with your own product and your own company, and then reflecting on jobs and humanity and AI working together. You can find all of that on our website.

Frank, to start things off, what’s your favorite example of GPT-3 so far? Mine is founding principles for a religion written in GPT-3. I’d love to hear your favorite and also your quick take on why the excitement — to start us off before we dig in a bit deeper.

Frank: My favorite out of the whole thing is it’s doing arithmetic. So, if you ask it what’s 23 plus 67, like just arbitrary two-digit arithmetic, it’s doing it. This is a natural language processing model. And so, basically, it got trained by feeding it lots and lots of text. And out of that, it’s figuring out — we think — how to do arithmetic, which is very, very surprising, because you don’t think that, like, exists in text. The excitement potentially is promising signs of, you know, progress towards general artificial intelligence.

So, today, if you want to do very highly accurate natural language processing, you build a bespoke model. You have your own custom architecture, you feed it a ton of data. What GPT-3 shows is that they train this model once and then they throw it a whole bunch of natural language processing tasks — like fill in the blank, or inference, or translation. And without retraining it at all, they’re getting really good results compared to finely-tuned models.

What actually is GPT-3?

Sonal: Before we even go into teasing apart what’s hype, what’s real, let’s first talk about the “it.” What is GPT-3?

Frank: So, we have two things. One, we have a machine learning model. GPT is actually an acronym — it stands for Generative Pre-Trained transformer. We’ll go through all those in a sec. But thing one is, we have a pre-trained machine learning model that’s optimized to do a wide variety of natural language processing tasks, like reading a Wikipedia article and answering questions from it; or guessing what the ending of a story should be; or so on and so on. So we have a machine learning model. The thing that people are playing with is an API that allows developers to essentially ask questions of that model. So, instead of giving you the model and you program it to do what you want, they’re giving you selective access via the API.

One of the reasons they’re doing this is that most people don’t have the compute infrastructure to even train the model. There’s been estimates that if you wanted to train the model from scratch, it would cost something like $5 to $10 million of cloud compute time. That’s a big, big model, and so, like, they don’t give out the model. And then two, the controversy around this thing when they released the first version was they were worried that if they just gave the raw model out, people would do nefarious things with it — like generate fake news articles that you would just, like, saturate, bomb the web — and so they were like, look, we want to be responsible with this thing, and so, we’ll gate access via API so then we know exactly who’s using it. And then the API can be a bit of a throttle on what it can and can’t do as well.

Sonal: Right. Well, while helping them learn. And just as a reminder, APIs are application programming interfaces. We’ve talked a lot about them on the podcast, and people who want to learn more can go to a16z.com/api to read all our resources, explainers. There’s so much we have on this whole topic. But the key underlying idea — and this goes to your point about the cost of what it would take if you were trying to build this from scratch — is APIs give developers and other businesses superpowers because they lower the barrier to entry — in this case, for anyone being able to use AI who doesn’t necessarily have a whole in-house research team, etc. And so, that’s one of the really neat things about the API.

But I do want to correct one misconception the folks out there aren’t aware of when it comes to GPT-3. What they’re describing as GPT-3, they’re actually playing with OpenAI’s API, which is not just GPT-3. Obviously, some of the technical achievements of GPT-3 are in the API, of course, but it’s a combination of other things. It’s like a set of technologies that they’ve released and it’s their first commercial product, in fact. So, that’s just to give people a little context on what the “it” is and isn’t there. Let’s go ahead and go a level deeper into explaining what it is. In their paper, they describe it simply as an autoregressive language model. Can you share what it is and kind of the category this fits in?

What categories does GPT-3 fit into?

Frank: Yeah. So, the broad category of things it fits into — it is a neural network, or a deep neural network. And architectures basically talk about the shape of those networks. At the highest level, visualize it as something comes in on the left, and then I want something to shoot out on the right side — and in between is a bunch of nodes that are connected to each other. And the way in which those nodes are connected to each other and then the connection weights, that’s essentially the neural network. GPT-3 is one of those things. Technically, it’s called a transformer architecture. This is an architecture for neural networks that Google introduced a few years ago. And it’s different than a convolutional neural network, which is great for images. It’s different than a recurrent neural network, which is good for simple language processing. The way the nodes are connected to each other results in it being able to do, essentially, computations on large sentences <Yes.> filled with different words and doing it concurrently instead of sequentially. So, RNNs, which were the former state-of-the-art on natural language processing, they’re very sequential. So, they’ll kind of go through a sentence a word at a time…

Sonal: Recurrent, right?

Frank: Exactly. These transformer networks can basically, sort of, consider the entire sentence in context while it’s doing its computations. One of the things that you classically have to do with natural language processing is you have to disambiguate words. “I went to the bank” — that could mean I want to go withdraw some money, or it can mean I went right up to the edge of the river — because we have ambiguity in these words. The natural language processing system needs to figure out, well, which sense of bank did you mean? And you need to know all the other words around that sentence in order to disambiguate it.

And so, these transformers consider large chunks of text in trying to make that decision all at once instead of sequentially. So, that’s what the transformer architecture does. And then what OpenAI has been doing is basically transforming this type of neural network, with the transformer architecture, on larger and larger datasets. Conceptually, think of it as you’ll have it read Wikipedia, and think of that as generation one. Generation two is, I’m going to have it read Wikipedia and all of the open-source textbooks that I can find. This generation, they trained it on what’s called common crawl. It’s kind of the same thing that Google uses to search and index the internet. There’s an open-source version of that. Think of it as — robots go onto every webpage, they gather the text, and now we’re using that as the training set for GPT-3.

Sonal: Yeah. Something like half a trillion words, I believe.

Frank: Yeah. It’s a crazy number of words. And then this thing has two orders of magnitude more than the previous attempts, that’s something like 175 billion parameters. For the purposes of this conversation, a way of measuring the complexity of a neural network.

Sonal: Right. GPT-2 had 1.5 billion.

Frank: And in between GPT-2 and 3, Microsoft did one that was 17 billion, right? So, like, there is a bit of an arms race here going on, which is, like, how big are your neural networks?

How GPT-3 learns

Sonal: What does it mean, because the paper’s called “Language Models are Few-Shot Learners.” And I remember this movement in one-shot learning where you can learn on very few examples, but honestly, what you just described to me sounded like almost a trillion examples, when you think about what it’s ingesting as an input. So, can you actually explain what few-shot even means in this context?

Frank: Yeah. So, first, they trained this model on the internet. Basically, what came in as input on the left side was reams and reams and reams of text — all the text they could get their hands on, and they cleaned it a little. And so, this is very traditional deep learning. It is not itself a zero-shot or a few-shot approach. It’s deep learned, which means I have incredible amounts of input text. What they mean in the context of this paper around no-shot and few-shot is, the model can perform a variety of natural language processing tasks. So, a good example of it is analogies — king is to queen, as water is to what, right?

In the context of the system, what you can do is you could give it an example of that, and they call that one-shot — which is, I’m going to give you an example of an analogy that’s completely filled out, and then I want you to fill out more analogies. Another task would be — pick the right ending of a story, and I will give you one example with the correct answer. So, I’m just going to give it to you once. Now, typically what happens when you do traditional neural network learning — you take an example, you give it to the system, and you tell the system the right answer. The system uses that right answer to basically readjust the neural net. It’s called backpropagation. And the theory is that, as it adjusts the weights inside the neural network, it will get that answer more correct the next time it sees it.

And so, everything up until this point has basically been — if I give you enough examples, I’m going to be able to tell whether that picture has a hot dog in it or not. I will be able to generalize the features of a hot dog, and I will basically deduce hot-dogness if you just give me enough pictures and you tell me, hot dog or not. What’s going on here is they train this model once, and then they give it one example — that example doesn’t adjust the weights of the model. It really just primes the system to basically prepare it to answer this type of question. So, you basically tell it, look — I want you to work on, fill in the blank, and I’m gonna give you one or a few examples (few-shot) of this, and then we’ll go from there. But those examples that you give it don’t adjust the weights of the model. It’s one model to rule them all. And this is kind of how humans learn. They don’t need to see 1,000, 10,000, 100,000 examples of hot dogs before they can start reliably telling whether it is a hot dog or not.

Sonal: It’s like how children learn language.

Frank: Yeah, exactly. Babies, before they can say cat and dog, can recognize the difference between cats and dogs — they didn’t see a million of them, right? In fact, they can’t say the words dog and cat yet. And so, maybe something like this is going on in the brain, which is you have this sort of general processor, and then it instantly knows how to adapt itself to solve a lot of different problems, including problems it had never seen before. And so, I’m going to go back to my favorite example of, like, what GPT-3 was used for. Like, how in the world did it deduce the rules for two-digit arithmetic by reading a lot of stuff? And so, maybe this is the beginnings of a general intelligence that can rapidly adapt itself. Now, look, I don’t want to get ahead of myself. It falls apart on four-digit arithmetic. And so, it’s not generally smart yet, but the fact that it got all of the two-digit addition and subtraction problems right by reading text, like, that’s crazy to me.

Fitting GPT-3 into the AI big picture

Sonal: The general takeaway is that it does some complicated things really well, and some really easy things really badly, and this is actually true of most AI. The researchers have a huge section on limitations where, “GPT-3 samples can lose coherence over sufficiently long passages, contradict themselves, and occasionally contain non sequitur sentences or paragraphs.” Now, of course, as an editor, that made me laugh because that’s also true of human writing. <laughter> So, I was like — okay, this is also true about the writing I’ve seen and edited, so I don’t know who’s talking here. Help me tease apart where we really are in this long arc. I’m having a hard time knowing what’s real, what’s not. Like, help me kind of understand what is this thing, really, at this moment in time.

Frank: So, we have the most sophisticated natural language processing pre-trained model of its kind. The natural language processing community has basically divided the problem of understanding language into dozens and dozens of sub-tasks. And task after task after task, GPT-3 goes up against the state-of-the-art, the best performing system. And basically what the paper does is lay out, okay, here’s where GPT-3 is approaching state-of-the-art, here’s where it’s far away from state-of-the-art. And that’s basically all we know, is — compared to state-of-the-art techniques for solving that particular natural language processing task, how does it perform? We’re really in the research domain. <Right.> So, if you were to ask me, can I build a startup on it? Can I build the world’s best chatbot on it? Can I build the world’s best customer support agent on it?

Sonal: I was going to ask you that.

Frank: Yeah, I think it’s really too early to tell whether you can build any of those things. The hope is that you could, and long-term, really, the hope is, having built a model like this and exposed an API, you could take any Silicon Valley startup that wants to solve a text problem — chatbots, or pre-sale support, or post-sales customer support, or building a mental health app that talks to you. All of those things will get dramatically cheaper and faster and easier to build on top of this infrastructure.

If this works, you have this generally smart system that’s already been trained, then you show it a couple examples of problems that you want to solve, and then it will just solve them with very high accuracy. All you have to do, as a startup or a programmer, is to say, “Hey, look, I’m going to give you a couple of examples of the type of problem that I want solved.” And then that priming is going to be enough for the system to get very accurate results. And, in fact, sometimes better results than if you had built the model and fed it the data sets yourself. So, that’s the hope, but we just don’t know yet.

Use and scalability

Sonal: That’s a really good reminder because they themselves are like, this is early days, it’s research, there’s a lot of work to be done — but it’s also really exciting, as you’re saying, because this is one of the most advanced natural language models we’ve seen. So, the question I have then, on the startup and building side — what would it take to — what are the kinds of considerations to make it more practical and scalable? I mean, for one thing, the size — you described how the transformer has this ability to sort of comprehend so much at once without doing it in kind of this RNN model, but the trade-off of that is that it’s so slow, or be able to fit on a GPU. So, I’d love to have a quick take from you on, what are the things that need to happen to make something like this more usable, etc.

Frank: I think what’s going to need to happen is that the OpenAI product team is going to have conversations with dozens and dozens of startups that are using their technology. And then they successfully refine the API and improve the performance, and set up the security rules and all of that, so that it becomes something as easy to use as say, Stripe or Twilio. Stripe or Twilio are very straightforward — send a text message or process this payment. This is a lot more amorphous, which is, “Hey, I can do SAT analogies. How’s that relevant for my startup?” Well, there’s a bit of a gap there, right? You have a startup that’s like, “Hey, I need my documents summarized,” or, “I need you to go through all of the complaints we’ve ever gotten and give me product insight for product managers.” And so, there’s basically a divide between there that needs to be closed over time.

Sonal: Right. So, what does this mean with the data world? Because one really interesting [thing] to me is, on one hand, APIs give you superpowers — kind of democratizing things. On the other hand, it kind of makes things a bit of a race to the bottom then, because then you have to differentiate — kind of private, proprietary, these other elements. So, do you have thoughts on what that means?

Frank: Yeah. I mean the hope for something like a GPT-3 is that it’s going to dramatically reduce the data gathering, cleansing, cleaning process — and, frankly, building the data model as well, your machine learning model. So, let me try to put it in economic terms. Let’s say we put $10 million into a Series A company, and then $5 million of it goes to getting data and cleaning it and hiring your machine learning people, and then renting a bunch of GPUs in Amazon or Google or Microsoft, wherever you do your compute. The hope is that if you could stand on the shoulders of something like GPT-3 — and it’ll be a future version of it — you would reduce those costs from $5 million to $100,000.

You’re basically making API calls and the way you program “this thing” is you just show it a bunch of examples that are relevant to the problem that you’re trying to solve. So, you show it texts where you had a suicide risk and you don’t need to show it a bunch because it’s pre-trained — and you show it a new text that it hasn’t seen before and you ask it, “What is the risk of suicide in this text exchange?” The hope is that we can dramatically reduce the costs of gathering that data and building the machine learning models. But it’s really too early to tell whether that’s going to be practical or not.

Sonal: So we know what it means for startups, but how do the incumbents respond in that kind of a world? But it seems almost inevitable that the big players — there might be an AWS potentially, right, that could, you know, make this a given in their services — like this kind of bigger question around this business model of AI as a service.

Frank: Yeah. So, the first thing I’ll say is this is OpenAI’s first commercial product, which is interesting, right? Recall that OpenAI started as a research institution, so we’ll sort of see what the pricing is. If this works, the scenario that I described earlier, which is — dramatically reduce the time it takes to build a machine learning inside product — then all of the public cloud providers and other startups will offer competing products because they don’t want to let OpenAI just take all of the, sort of, text understanding ability of the internet, right?

Google Cloud and Microsoft and Amazon and Baidu and Tencent, like they’re all gonna say, “Hey, look, I can do that too — build your application on me.” Now, I will say that because of the large costs of training the model — so I’d mentioned estimates ranging from $5 million to $10 million to train this thing once — and obviously, they didn’t train it once to get to where they were, they trained it multiple times as they did the research process. And so, this is not going to be for the faint of heart. It’s going to come on the back of a lot of money with very skilled scientists using enormous infrastructure. But to the extent that this product works, then you’re going to have very healthy competition among all of the incumbents. You might even have new players who’ll figure out a different angle on it.

Working with machine learning

Sonal: You know, it’s really fascinating watching the people who have access. And basically, the recurring theme is that it’s not like plug and play, it’s obviously not built and ready for that yet. The prompt and the sampling hyper-parameters matter a lot. Priming is an art, not a science. So, I’m curious for where you think the knowledge value is going to go in the future. What are the sort of — the data scientists of the future going to look like for people who have to work with something like this? Now, granted the models are going to evolve, the API will evolve, the product will evolve — but what are the skills that people need to have in order to really do well in this world coming ahead?

Frank: It’s really too early to tell, but it is a fundamentally different art of programming, right? So, if you think of programming to date, it’s basically — I learn Python, and I learn to be efficient with memory, and I learn to write clever algorithms that can sort things fast. That’s well-understood art, thousands of classes, millions of people know how to do that. If this approach works, basically, there is this massive pre-trained natural language model, and the programming technique is basically I show you a couple of examples of the tasks that I want you to perform — it’ll be about what examples do I show you, and in what form? And do I show you the outliers, or do I show you some normal ones, right? And so, if this approach works, it’ll all be about — how do you prime the model to get the best accuracy for the real-world problems you actually want your product to solve? Programming becomes — what examples do I show you, as opposed to how do I allocate memory and write efficient search algorithms? It’s a very different thing.

Sonal: Vitalik Buterin, the inventor of Ethereum, described this when he was observing some of this buzz around GPT-3 that, “I can easily see many jobs in the next 10 to 20 years changing their workflow to ‘human describes; AI builds; human debugs.’” There’s a lot of speculation about how this might affect jobs. It can displace customer support, sales support, data scientists, legal assistants, and other jobs like that are at risk. Do you have thoughts on the labor and jobs side of this — like just sort of the broader questions and concerns here?

Frank: The way I think about this generally — and informed a lot by Erik Brynjolfsson and other people — so if you think about a job as a set of tasks, some tasks will get automated, and then some tasks will be stubbornly hard to automate, and then there’ll be new tasks. And so, think of jobs as sort of an ever-changing bundle of tasks, some of which are performed by humans today, some of which will get automated, and then there are new tasks. And so what Vitalik describes — if this AI stuff works, being able to prime the AI system with the right examples, and then being able to debug it at the end — those are two new tasks. No human on the planet gets paid to do that outside of AI researchers today. But that could be mainstream knowledge work in 10 years, which is — you pick good examples, and then you debug it at the end. So, you have these brand new tasks that are generating economic value and people get paid for them, that didn’t exist before.

Sonal: I find it very fascinating what you said, by the way, because what it also means to me is it becomes more inclusive for more people to enter the worlds that might have been previously closed off to a certain class of type of programmers, or people who have certain technical skills, because — let’s say you’re very good at describing things, and it’s more of an art than a science, and you’re very good at sort of fiddling with and hacking at things, you might be better off than someone who went through like years and years of elite Ph.D education at tuning something than someone else.

Frank: I think the machine learning algorithms will invite more people who would otherwise be discouraged into pursuing careers, in careers they wouldn’t have naturally risen to the top of. So I think you’re right.

Ethical concerns and safeguards

Sonal: What do you make of the concern — there was concern that GPT-3, these answers that it gave, that it predicted, were rife with racism or stereotypes. What do you make of the data issues around that?

Frank: Okay. We’re going to feed it every piece of text on the internet and then we’re going to ask it to make generalizations. What could possibly go wrong? A lot could possibly go wrong. If you look at the heart of this system, it’s basically, I’m trying to guess the next word. And the way I make my guess is, I go look at all the documents that have been written ever and I ask, what words are most likely to have occurred in those documents, right? You’re going to end up with culturally offensive stereotypes. And so, we need to figure out — how do we put the safety rails? How do we erect the APIs? I’m glad the OpenAI researchers and the community around them are being very careful about this because we obviously have to. How do we basically teach it the social norms we want it to emit, as opposed to the ones that it found by reading text?

Sonal: Another whole philosophical sidebar, but really important is, if you think about the internet as the sum total of human knowledge, then other things that reflect many of the realities in the world, which are atrocious and awful in many cases. The flip side of it is, it’s a lot harder to change the real world and people and behavior and society and systems, but probably a hell of a lot easier to change a technical system and be able to do certain things. So, to me, what’s implicit in what you said is that there’s actually a solution — I don’t mean to be solutionistic, but that’s within the technology that you don’t necessarily get from IRL, in real life.

Frank: Yeah, that’s exactly right. And if it were in algorithm land, so to speak, where we are, right, GPT-3 and its descendants — let’s say GPT-17 gave you a text document, right? It wrote a text document for you. You could take that document and put it through whatever filter that you wanted, right, to filter out sexism or racism, and that layer could be inspectable and tuneable to everybody. You didn’t know how GPT-17 came up with its recommendations, but you have this safety net at the end, which is you can filter out things that you don’t want. So, you have the second step that you can actually put into your system. You don’t have to depend just on the first thing, you can catch that at a subsequent stage.

Updating the Turing test

Sonal: Right. And you can have sort of a system of checks and balances. So a broad meta question — one of my favorite posts was from Kevin Lacker, and he basically gave GPT-3 a Turing test, and he tested it on these questions of common sense, obscure trivia, logic. And one of the things he observed is that, “We need to ask it questions that no normal human would ever talk about.” And so, he said, if you’re ever a judge in a Turing test, make sure you ask some nonsense questions and see if the interviewee responds the way a human would. Because the system doesn’t know how to say I don’t know, and this goes at this question of what does a Turing test tell us? And there’s been a lot of work, as you know, over the years about the modernization of the Turing test — like in 2016, Gary Marcus, our friend, Gary Marcus, Francesca Rossi, and Manuela Veloso published an article “Beyond the Turing Test” in “AI Magazine.”

Barbara Gross of Harvard wrote a piece called “What Question Would Turing Pose Today?” in “AI Magazine” in 2012. And she basically starts by saying that in 1950, when Turing proposed to replace the question “can machines think?” with the question “are there computers which would do well in the imitation game?” — at the time, computer science wasn’t a field of study. You know, Claude Shannon’s theory of information was just getting started. Psychology was just only starting to go beyond behavior. And so, what would Turing ask today? He’d probably propose a very different test. And so, the question I really wanted to ask you is, how do we know if the thing is measuring what it’s supposed to measure, or answering what it’s supposed to answer, or that it’s getting smarter, I guess?

Frank: This is more a philosophical question than an engineering question. So, why don’t I say what we know, and then I’ll widely speculate on the other stuff?

Sonal: That’s great. That’s life and science, so go for it.

Frank: Exactly. So, basically, if you read the paper, you’ll see that it compares GPT-3’s performance against various other state-of-the-art techniques on a wide variety of natural language processing tasks. So, for instance, if you’re asking it to translate from English to French, there’s this thing called the BLEU score. The higher the BLEU score, the better your translation. And so, every test has its measure. And so, what we do know is we can compare GPT-3 performance versus other algorithms, other systems. What we don’t know is, how much does it really understand? So, what do we really take away from the fact that it aced two-digit arithmetic? Like, what does that mean? What does it understand of the world? Does it get math? Let’s say you had a system that was 100% accurate on every two-digit arithmetic problem that you ever gave it. It’s perfect at math, but it doesn’t get it. Like, it doesn’t know that these numbers represent things in the real world, but what does that mean to claim that it doesn’t get it? That’s a philosophical question.

Sonal: Right. It’s philosophical because the question then becomes — does it even matter if it comes to applying things practically? Because I think about this from the world of education, you know, there’s a big focus on metacognition and the awareness of knowing what you know and don’t know. But at a certain point, if the kid is doing well on the test and the test is applicable to the world, and they can basically survive and do well, does it even matter if they really understood what arithmetic really means, as long as they can solve the problem when you go to the store, that I give you a dollar, I get 5 cents change back? You know what I mean?

Frank: That’s exactly right. And if you generalize that out to other tasks that humans solve in the real world, imagine you just got good at 100 and then 1,000 and then 10,000 of these tasks that you have never seen before. Let’s say descendants of GPT-3 got that good at a wide variety of language tasks — what does it mean to insist, but it doesn’t get the world, it doesn’t get language, right? <laughter>

Predictions about future implications

Sonal: Yeah. That’s fantastic. I’d love to get sort of your perspective on how we think about this broader arc of innovation that’s playing out here. Daniel Gross called GPT-3 screenshots the TikTok videos of nerds. And there’s something to that — it’s kind of created this inherent virality. So, I’m curious for your take on that. On the one hand, some of the most important technologies start out looking like a toy. Chris Dixon paraphrased a really important idea from Clayton Christensen about how disruptive innovation happens. But a lot of the people who are researchers really emphasize — this is not a toy, this is a big deal.

Frank: There are a lot of TikTok-ish-like videos that are coming out of the whole playground, which is basically a place where you can try out the model. And on the one hand, people are saying it’s a toy because they’re in the sandbox and they’re basically having fun feeding it prompts. Some of those examples are actually really good, and some of those are, like, comically bad, right? So, it feels toy-like. The tantalizing prospect for this thing is that we have the beginnings of an approach to general intelligence that we haven’t seen us make this much progress on before, which is — today if you wanted to build a specific system for a specific natural language processing task, you could do that. Custom architecture, lots of training data, and lots of hand-tuning and lots of, like, Ph.D time.

The tantalizing thing about GPT-3 is, it didn’t have an end-use case in mind that it was going to be optimal for, but it turns out to be really good at a lot of them, which kind of is how people are. You’re not tuned to, like, learn polka or double-entry bookkeeping, or learn how to audio-edit a podcast — like, you didn’t come out of the womb with that, but your brain is this general-purpose computer that can figure out how to get very, very good at that with enough practice and enough intentionality.

Sonal: Well, it’s really great that you use the word tantalizing because if you remember the Greek myth root behind it, Tantalus was destined to constantly get this like tempting fruit dangling above him as punishment. And it was so close yet so out of reach at the same time. So, bottom line it for me, Frank.

Frank: It’s tantalizing, right? Now, look, there’s a limit to how big these models can get and how effective the APIs will be once we sort of, you know, unleash them to regular programmers. But it is surprising that it is so good across a broad range of tasks, including ones that the original designers didn’t contemplate. So, maybe this is the path to artificial general intelligence? Now, look, it’s way too early to tell. So, I’m not saying that it is, I’m just saying it’s very robust across a lot of very different tasks, and that’s surprising, and kind of exciting.

Sonal: Thank you so much for joining this episode of “16 Minutes,” Frank.

Frank: Awesome. Thank you, Sonal, for having me.

• 

  Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

  Follow Twitter
• 

  Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

  Follow Twitter

In this special “2x” episode (#32) of our news show 16 Minutes — where we quickly cover the headlines and tech trends, offering analysis, frameworks, explainers, and more — we cover the tricky but important topic of Section 230 of the Communications Decency Act. The 1996 law has been in the headlines a lot recently, in the context of Twitter, the president’s tweets, and an executive order put out by the White House just this week on quote- “preventing online censorship”. All of this is playing out against the broader, more profound cultural context and events around the death of George Floyd in Minnesota and beyond, and ongoing old-new debates around content moderation on social media.

To make sense of only the technology and policy aspects of Section 230 specifically — and where the First Amendment, content moderation, and more come in — a16z host Sonal Chokshi brings on our first-ever outside guest for 16 Minutes, Mike Masnick, founder of the digital-native policy think tank Copia Institute and editor of the longtime news & analysis site Techdirt.com (which also features an online symposium for experts discussing difficult policy topics). Masnick has written extensively about these topics — not just recently but for years — along with others in media recently attempting to explain what’s going on and dissect what the executive order purports to do (some are even tracking different versions as well).

So what’s hype/ what’s real — given this show’s throughline! — around what CDA 230 precisely does and doesn’t do, the role of agencies like the FCC, and more? What are the nuances and exceptions, and how do we tease apart the most common (yet incorrect) rhetorical arguments such as “platform vs. publisher”, “like a utility/ phone company”, “public forum/square” and so on? Finally: how does and doesn’t Section 230 connect to the First Amendment when it comes to companies vs. governments; what does “good faith” really mean and what are possible paths and ways forward among the divisive debates around content moderation? All this and more in this 2x+ long explainer episode of 16 Minutes.

Show Notes

• An explanation of what Section 230 is and what it covers [2:03]
• Publishers vs. platforms and a discussion of current events [6:37]
• Why platforms are not legally considered public utilities or “public squares” [11:57]
• An overview of the Executive Order on Section 230, and the powers of the FCC [18:10]
• How the Executive Order restricts federal spending on platforms [27:13]
• The difficulty of content moderation and why Section 230 protects all websites [30:23]

Transcript

[updated intro as of January 8, 2021 here]

What is Section 230?

Mike: The law is actually very short, and very simple, and very straightforward. And I should note, that the Communications Decency Act itself did have many more things that it did, but all of that was determined to be unconstitutional. So the only thing that survives is Section 230. There was a big lawsuit, ACLU vs. Reno, in the late 90s, and that threw out most of the Communications Decency Act as unconstitutional; the thing that remained was 230.

So Section 230…really does two things, and they’re somewhat related, and they’re both incredibly important to the functioning of the modern internet. The first thing that it does is it puts the liability on the person actually violating the law. So, if someone goes onto a website, and says something that is defamatory or other otherwise violates the law, the liability for that action belongs on the person who is speaking — and not the platform or site that is hosting that content.

The second thing that it does is that if a website chooses to moderate its content (or anything that is put on the site), then it is not liable for those moderation choices.

Sonal: I’m so glad you’re bringing that up because this is the #1 thing I wanted to start with, which is, the flip side of it — not just the protection, but the fact that they can moderate whatever they want — so can you actually break that down, Mike? What does that mean?

Mike: So, where it came from — which I think is important to give sort of the history very quickly — is that, there were a series of lawsuits in the early 90s that tried to hold internet services that had moderated some content. There were defamation cases, effectively, brought up.

The most famous one is Stratton-Oakmont vs. Prodigy and as a little fun aside, Stratton- Oakmont was a financial firm that was immortalized in the movie “The Wolf of Wall Street” —

Sonal: That’s a fun fact.

Mike: Yes. And Stratton-Oakmont got upset because people in Prodigy’s message boards were accusing the company of being up to no good and so they sued Prodigy.

A court said that Prodigy was liable for the libelous statements because Prodigy positioned itself as a family-friendly service that would moderate content. Because it moderated some content — [i.e. taking] down cursing or porn (or anything that it felt was inappropriate) — anything left up (according to the judge) [Prodigy] was now liable for as if it had written that content.

And that freaked out people in Congress namely…two members of the House: Chris Cox (a Republican) and Ron Wyden (a Democrat). They put together Section 230 to say wait, that’s crazy. If a website wants to moderate content to create, for example, a family-friendly environment, it shouldn’t get sued for the content that it chose not to take down.

And so that section of CDA 230 is designed to make sure that any website can moderate content how it sees fit, in good faith, to present the content in a way that meets with the goals of the service.

Sonal: Right. And to be clear, these are not just quote “content moderation things.” It could be spammy posts [or] the kind of thing that would actually turn you off from using the service [i.e.] family-friendly site getting rid of porn. The companies can use whatever discretion they wanted, as long as it complies with their terms of services, which could change.

But what’s interesting about this back-story: It’s a very small thing that was preserved, but that had huge consequences for where we are today, in terms of the internet we have today. Whether it’s going on a recipe-swap site; whether it’s sharing photos of family and friends; whether it’s posting a car for sale — there are so many layers to this. It has allowed the modern internet to thrive. One of the best lines I heard (I think this was actually in Verge) is that, in many ways, this Act was a gift not to big companies, but a gift to the internet.

Mike: I think the point is not that it is the biggest gift to big internet companies OR that it’s the biggest gift to the internet — I think it’s really the biggest gift [of] free speech for everybody, right? Because if you don’t have 230 set up the way it is set up, there would be much more limited ability for users to actually post content online.

It’s a little bit crazy to me that people think that changing or getting rid of 230 will enable more free speech, when the balances that are set up within 230 are very much designed as a gift to free speech.

Platforms vs. publishers

Sonal: Okay, so now my question for you is, given that we did enter this world where user-generated content — whether on sites like YouTube with videos, educational or non-educational, political or non-political — we now live in a world where…people often use the framing of “platform versus publisher” (which I think is kind of meaningless and arbitrary). Sometimes, [they also] use the ridiculous phrase, “platisher,” as a hybrid of the two. I’d love to get your take on that framing and how that doesn’t (or does) apply here?

Mike: So, one of the things that comes up over and over that people say is, “Well if they moderate or if they change content, they are no longer a publisher; they are now a platform, and therefore, they lose Section 230 protections.”

The law makes NO distinction between platform and publisher; the law is not designed to protect one or the other, or say that there is difference between it — there’s no classification. It’s not a safe harbor where you have to meet, you know a, b, and c criteria in order to get the protections. You just need to be an “interactive computer service” that hosts third party content.

So the debate over “Are they a publisher, or are they a platform?” is completely meaningless under the law.

Sonal: Let’s actually talk about some recent events because I think it’s a useful case in sort of understanding 230 — and then we can break down some of the recent news as well around that.

So one recent event is that Twitter added a feature earlier this week where, [on] one of the President’s tweets, they added a link to other sites as a sort of quote “fact-check” mechanism. This could be contentious because a lot of people do not actually believe that everything the media writes is correct. That said, it linked to other third-party news sites, and it kind of labeled it as a fact-check feature.

Then they added another thing where they kept a tweet from the president up — in the context of the Minnesota George Floyd protests — but put like a limit on it where people could retweet with comment but they couldn’t retweet, like, or reply to it, because it violated their site’s terms of services around speech that incites violence.

And so, in case one, they were adding what they quote called a “fact check” layer; in case two, they were adhering to their own terms of service around spreading violent speech, which said they kept up in the public interest.

So that’s a super, super high-level summary of what happened so far. My question for you now, Mike, is how [Section 230] does and doesn’t apply here? Because in this case, “the fact check” could be construed as commentary content, not just third-party content.

Mike: It’s a really complex topic. Each layer of it adds new complexities and each of those complexities are in some way important.

Let’s do the two tweets separately. The first tweet, they added something — and just as a minor correction (and this has been going around a lot; people said this is the first time that Twitter had used this) — Twitter has been using that feature over the last couple months but, this is the first time they have done it on a politician’s tweet.

What’s amusing to me is the time I saw it used…about two weeks earlier. It was used to debunk a Jimmy Kimmel video that was making fun of Mike Pence. Twitter put on a thing that said this is manipulated media, it is not accurate. It was a tweet that had gone viral. It was making fun of something that Pence had done, and Twitter stepped in and said no, this is incorrect or manipulated media. [Twitter then] had a link to third party content saying why it was manipulated.

And so, that is allowed under 230. What it is doing is adding more speech — it is linking to other sources [and] providing more context. The part that is not protected by 230, was never protected by 230, and no change to 230 is going to change that, is any speech that comes directly from Twitter itself. So, in this case, that was the very narrow line that was put under Trump’s tweet. It said something like, “Get more facts about mail-in ballots,” or something to that effect. That particular line is from Twitter itself and therefore is not protected by 230. But, it IS completely protected by the First Amendment. The third-party content that they link to is then protected by 230.

[In] the second tweet, Twitter did something new (which I had not seen before) in which they put up a note that said that this tweet violates the terms of service; however, they want to keep it up because they feel that [the tweet] is relevant and important for people to see [its] content but to understand that it violated the terms of service. So they added more context and limited the ability for people to retweet or reply to it.

And again, this is 100% allowed by 230. It did not remove the content, it didn’t take it down. Even if Twitter chose to take it down, or take down his speech, or take down that tweet, that wouldn’t violate his free speech rights. The First Amendment protects people from the government acting, not from a company. Now, I have also since seen Twitter use “This tweet violates our terms of service, but we are leaving it up because it is newsworthy,” message on at least one other tweet this morning from somebody who was defending the president.

Platforms as a public square

Sonal: Let me ask you another question, and then we can break down the executive order…Since we already debunked this platform-publisher distinction, [what do you make of] these companies that provide these interactive web services are like phone companies? They always use this line that, “Oh but imagine if the phone company decided to take down that conversation you had and interrupted you in the middle?” What do you make of that analogy?

Mike: Yeah so that is popular in a wide variety of circles across the political spectrum and doesn’t fall into any sort of partisan viewpoint — sort of the “public utility” argument.

Sonal: And that by the way, of course reminds me of Net Neutrality, which we both covered quite a bit.

Mike: Right. There are some funny parallels between this situation and Net Neutrality in that a lot of people’s positions are reversed from one to the other.

Sonal: We don’t have the time to talk about Net Neutrality, but I covered it extensively at WIRED, as you know, and from all different perspectives: from carriers, to FCC, to internet companies, you name it. That is exactly what’s fascinating to me, is that the positions and the sides are inverted in this case. So, anyway, what do you make of the phone and common carrier-type argument?

Mike: So it’s an important one to understand, but I don’t think it applies. I think that most people who are deeply familiar with public utilities and what is required to be declared a public utility would recognize that internet services — what’s sometimes called “edge providers” (which are the services that you and I use every day, that we interact with) — do not qualify and do not meet the requirements of a typical traditional utility service. To clarify, what that means [is that] usually a utility service is something that is offered to everybody, but is also commodified. If you use AT&T, or Sprint, or Verizon, you are getting the exact same service. There is no real differentiation in terms of the service that you’re getting — it is a commodity, one provider to the other. Same thing.

That is not the case with various internet edge providers i.e. Google, YouTube, Twitter, Facebook. Each of them have all of these different features and all these different things. They are not 1:1 replaceable. It is not a commodity that you can switch out; therefore, the public utility argument does not really apply.

You can argue that there should be some other kind of classification (and some people do argue that), but comparing them directly to a telephone service is different because it’s not core infrastructure — [they’re] things that are at the edge, things that you use as a service provided beyond that.

Sonal: What do you then make of this “public square / public forum” argument?

Mike: People say that Twitter or Facebook shouldn’t be allowed to do any moderation or take down any content because it’s the new public square, and therefore violates their rights. They will often point to two different lawsuits in making this argument: one is Pruneyard, the other is Packingham. These two cases… have been brought up in a whole bunch of lawsuits and I’ll just say that, every time they’ve been brought up in a lawsuit to argue that a social media site is the public square, they have failed. I have not seen a single judge anywhere agree that these things make any sense in this context.

But just to give the quick background on the two cases, and they go deep, but I’m going to give as high-level as I can, and as quick as I can: Pruneyard was a case about a mall that was trying to kick people out, effectively. It was argued that the mall was a gathering place and became the sort of de facto public square and that took away some of the rights of the private property owner [of the] mall to kick people out. The court said that it was a de facto public square and they could not kick people out.

Now it is an extraordinarily limited ruling and extraordinarily focused on the facts of that case. [The mall] was effectively the only place in town that anyone could gather. The mall owner sort of acted as a local government and was therefore replacing government functions — functions that normally were done exclusively by the government. Every other case after that that references Pruneyard has effectively limited it; it only applies in a very, very narrow situation, which is basically Pruneyard and Pruneyard alone. You can’t just say that something is a public square.

The Packingham case is a more recent case. It was a Supreme Court case that kicked out a state law that basically said if criminals had done some sort of criminal activity online, part of their punishment could be that they are barred from using the internet. The Supreme Court said you cannot pass a law that kicks people off of the internet because the internet is so essential to people’s lives and ability to work and all that kind of stuff. So people have taken that to mean [that] the services themselves cannot kick people off, but that is not what the case has said — it just said that the government cannot pass a law that forces people offline.

There is a third case that people never mention but is the most important case. It was just decided last summer, and that is the Manhattan News Network case. I won’t get into the details but the Supreme Court ruling was written by Brett Kavanaugh, who was the most recent appointment, and his ruling said that you can’t just declare any place where people can speak — even if a lot of people speak there — a public square. It doesn’t become a state action; it doesn’t take on government control.

The idea that something is a public square or that there is state action involved from a private company only applies in a very limited set of circumstances where that service or operation is, again, replacing activities that were traditionally done by the government. The ruling makes it very clear that Twitter, Facebook, YouTube, and every other website out there does not qualify. They are not replacing government [and] are not offering services that were traditionally given by the government.

Sonal: Right. This basically means that if the sites DO perform services that are exclusively a service provided by the government (i.e., if the government decided that all tax reimbursement would be done entirely online and no longer through the U.S. Postal Service), they would then have to comply [with] provisions.

Mike: Right, there could be an example, something that was traditionally and exclusively handled by the government. I could see an argument where someone could not be kicked off or blocked because that would imply state action issues.

Overview of the Executive Order

Sonal: So now let’s talk about the news — again, as a way to explain what CDA 230 is and isn’t. We’ve explained and debunked some of the myths and framings around arguments of platforms vs publishers and analogies to phone systems. Let’s talk very briefly about the recent executive order that was issued this week…[and discuss] what the executive order can and can’t do here, or what it purports to do and doesn’t do.

Mike: There were drafts of this executive order that made the rounds over the last two years. This is something that the White House has been thinking about. I reported on it, a number of other news sites reported on it as different drafts were leaked out to the press about earlier versions of this executive order. And, the story is that, in the past, they’ve passed this around to different agencies like the FCC and the FTC. The message that the White House got back was that this was unconstitutional, and they couldn’t do any of this, but it seemed that they took it out of the drawer, dusted it off, and then put a fresh coat of paint on it.

[The executive order] says a lot of very angry stuff about the internet services and platforms and the way they handle moderation. There are seven different sections but [there are] two sort of scary-ish parts of the executive order; to one extent, the order effectively tasks the FCC with coming up with a new interpretation of 230 [but] hints very strongly what the FCC’s interpretation should be — that interpretation is totally at odds with both what is written in the law and what 20 years of case law have said.

That’s worrisome only to the extent that anyone would ever actually pay attention to that FCC interpretation. The FCC in ACLU vs. Reno (which is the lawsuit that rejected and made most of the Communications Decency Act unconstitutional) made it extremely clear that the FCC has no authority whatsoever to regulate websites. None, zero, zilch. It’s not even an open question — they cannot.

Sonal: And just to be very clear here, the FCC (Federal Communications Commission) is an independent agency; it has a five-member commission. I believe there’s currently three Republicans, two Democrats.

Because it comes up a lot what FCC can do / can’t do, like, it cannot make laws, but it does have the ability to interpret existing laws and put out certain rule making things. They do these Requests for Comments which create public records of people’s commentary and whatnot. They also have the power to ask for documents — and they can do distracting things — but they may not have legal-making authority. So I think it’d be very helpful for you to break down a bit more specifically what they can get away with and also can’t.

Mike:  So they can do rule making, and that is a long involved process. Interestingly, because [the FCC] is an independent agency, the President cannot instruct them to do something. The executive order instructs the NTIA (which is part of the Commerce Department) to ask the FCC to do this. Technically, the FCC does not need to do this, but the FCC will certainly feel the pressure to probably do something. The FCC could certainly create a lot of a lot of nuisance, and yes, there will be comment periods, and people’ll have to testify, and put in comments. As we saw with the Net Neutrality hearing, the comment system was filled up with bots and nonsense, so the commenting and the rule making process is a bit fraught with distraction.

And so yes, [the FCC] can make rule making and can do something to enforce that rule making. If the rule making covers things that it is authorized, that the FCC is authorized to have regulatory power over by Congress —

Sonal: That are in its jurisdiction, so to speak.

Mike: That are in its jurisdiction — and websites are clearly not. Congress has never said that websites are within the FCC jurisdiction, and the main court case that tested the theory that websites were in the FCC jurisdiction has said no.

One other thing that I do want to note about the executive order and the request to the FCC is that, it is couched in a term that totally misinterprets CDA 230.

Sonal: Which is?

Mike: So earlier, I talked about the two different parts of the CDA — that one is about liability on third-party content, and one is about the platform’s protection in moderation. There are a few very narrow conditions on that moderation ability. They say: it has to be in good faith; and there’s a list of different content that you can moderate that includes otherwise objectionable content.

That otherwise objectionable content is very, very broad — it can cover basically whatever the platform thinks is otherwise objectionable. In order to argue good faith, [you] would open up a whole other First Amendment can of worms.

But what the instructions to the FCC indicate is that those limitations — the good faith, otherwise objectionable stuff — somehow applies to the first part of CDA 230, which is the part about not being responsible for third-party content. That has never been the case. Nobody’s ever suggested it is the case. It has never shown up in any lawsuit. It has never been argued in a legitimate way and yet, the executive order suggests that the FCC should look into whether or not that interpretation makes sense.

Sonal: So you’re basically saying that the two provisions of CDA 230 — that people are not liable for libelous content that their users might put on their site (or any other content their users might put on their sites) — is being conflated in this case with the good-faith aspect of being able to discretionarily moderate in “good faith.”

Mike: Exactly; they’re sort of mixing those two things up. I would argue that is done in bad faith, to make use of the good faith, limitation on all this.

Sonal: Oh, man. Right. What other aspects of the executive order — again, without going into breaking down every little detail because this is really more about the underlying principles — would you say have impact for understanding and really interpreting and explaining what CDA 230 is and isn’t?

Mike: So one important part — and this was added at the last minute, perhaps literally, because the draft that was leaked the night before did not have this, but the final executive order did have it — is that, it instructs the Attorney General to draft a law — oddly, not a federal law, but to draft like a reference state law — to effectively reinterpret CDA 230 in a way that diminishes its power.

And that could be problematic. Here’s an aside that I probably should have brought up earlier: 230 is not a universal immunity. It is not as universal as people make it out to be. One thing that it does not cover is federal criminal liability. So, if you break a federal law — drug trafficking, human trafficking —

Sonal: Child pornography, etc.

Mike: Child pornography, all of that stuff — the sites are still liable; 230 specifically exempts that. So, the Justice Department and the FBI, if they felt that any of these platforms were violating federal law, they have always, always under 230 been able to go after those sites and that includes third-party content. There are a whole bunch of conditions on that.

So if there is drug dealing, human trafficking going on those sites, [they] potentially could be criminally liable. The Attorney General, and the Justice Department, and the FBI have always had only way to make use of the law to go after these sites. And yet for the last few months, the Attorney General has been attacking 230 and acting as if it limited his power in some way when it simply does not. But now he can draft a law, and, he’s sort of already been doing that.

Sonal: What’s so amazing about what you just said, though, Mike, the part about the federal part actually immediately reminded me of the encryption debate — which we actually have discussed on this very show “16 Minutes” (and listeners can listen to our reframing of that debate); another place where policymakers on both sides have very conflicted views on.

Mike: Yeah, and there’s already a bill that’s in Congress — that was put together with the help of the Attorney General — and it sort of ties the 230 debate to the encryption debate. And, it’s very convoluted.

Sonal: Oh, this is the EARN IT–

Mike: The EARN IT Act. And what it has the potential to do is to say that if you are offering end-to-end encryption on your service, you no longer get 230 protections (it’s a little more complicated than that); but, his abilities to do that in a manner that would remain constitutional is a pretty big question.

But again, it could create a huge nuisance. And part of this is also [the Attorney General is] going to establish a working group so there will be discussions, roundtables, panels, hearings, subpoenas and all sorts of things that are going to happen in the meantime that are designed to be an intimidation tactic to try and — the phrase that everyone uses is “work the refs”, right? Tt means basically, “Hey, Twitter/ Facebook/ YouTube, if you don’t want us to keep causing trouble for you, maybe don’t be mean to us,” you know. Don’t fact check us. Don’t limit our tweets. Don’t limit our content. Don’t put extra notices on it or other limitations on it because the more you do that, the more of a pain we’re going to be to you.

Restrictions on federal ad spending

Sonal: To summarize, at a super high level, the FCC has extremely limited jurisdiction over websites, specifically; the Attorney General does have some ability.

We haven’t talked about what’s not in the executive order, but this is where there’s a little bit of the dust storm [that] is very distracting which is that Congress could choose to rewrite policy (if they wanted) using this as an incitement for that.

Mike: Yeah. There are people in both the House and Senate who have said that they will introduce legislation based on this and try and do more than the executive order can do. Whether or not that legislation can actually go anywhere, any such legislation would almost certainly be subject immediately to a First Amendment challenge and would likely fail, but that would be many years into the future.

Sonal: Right. So we forgot one bit of the executive order, which is probably the only legit thing in there seemingly, which is that part of this had the threat of limiting any government dollars of advertising going to these sites. And I by the way did a little quick check and based on federal procurement records (this is according to The Verge) apparently, only $200,000 of advertising have been provided to Twitter specifically since 2008 — which sounds a little crazy to me. It can’t be getting everything, that seems way too low but even still, it does suggest that the government advertising is actually a very tiny piece of the bottom-line revenues of these companies. But, I’m curious for your take on that.

Mike: Yeah. So, that is one thing that an executive order actually can do, right, which is instruct certain federal agencies in terms of how they’re spending their money in some form or another —

Sonal: — Oh by the way, to be clear, when you say “their money,” we’re actually still talking about taxpayer money here.

Mike: Yes, yes — mostly taxpayer money. There are a few exceptions, but mostly taxpayer money is what we’re talking about here. What’s funny is the executive order sort of implies that it is telling agencies to stop spending on these websites, but it doesn’t actually say that. It says they have to account for what they’re spending, and they have to submit it to the Office of Management and Budget, and then something maaay happen in the future based on that. And the implication is that they should not be spending.

So, there could be a tiny, tiny, tiny miniscule drop in spending and, what’s silly of course is that, I would bet that the various political campaigns of everyone who is cheering this on are still spending much more money themselves as campaigns on these social media platforms in order to advertise.

Sonal: No question! On all sides.

Mike: So the one concern from a societal perspective is that the few federal agencies that do advertise on social media, actually probably have pretty good reason for that, and the one big example is the Census Bureau. And it’s 2020, and we’re in the midst of supposedly collecting the census.

Sonal: I forgot about that, right.

Mike: Because every 10 years, we have to do a census.

And one of the best ways that the government has found to get out the word, and to get people to actually fill out their census forms is through advertising on social media; therefore, pulling that budget and telling the Census Bureau that they cannot advertise actually could limit the ability of the Census Bureau to collect the data that they are required under the Constitution to collect.

The difficulty of content moderation

Sonal: So, Mike, this is a wonderful summary so far of what Section 230 of the Communications Decency Act does and doesn’t allow; of the recent news, what’s hype/ what’s real, and sort of really using that to explain these laws that have allowed our modern internet. I will be linking — just in the show notes so people know — to a lot of the articles that did good explainers, a lot of your wonderful pieces in particular, as well as the actual executive order, and the analysis of the differences that Eric Goldman (our mutual friend) put up.

One question I do have for you — this is very much playing out against a broader backdrop of debates around big tech, debates around content moderation — is, given that the recent example did not necessarily remove or necessarily even fully restrict (except maybe in spread and engagement and scale), there [have] been a lot of complaints about things like shadow banning. There’s also a lot of conflation between content and behaviors (like what sites can do versus what they say) and for me, it seems like when it comes to this content moderation debate, you’re damned if you do and you’re damned if you don’t.

I’m curious for your thoughts on a) where this fits in that longer/broader scape of that debate; and then b) is there a way forward in your mind?

Mike: So I put a joke on Techdirt a few months ago and I keep referring to it over and over again. There’s a famous economist, Kenneth Arrow, he had this thing called the Arrow impossibility theorem. He looked at all different kinds of voting systems and argued that none of them can accurately reflect the will of the populace. And so I did a play on that, which I called — humbly — the Masnick Impossibility Theorem.

Sonal: You are a very humble guy. We go way back, I think it’s been quite a number of years I’ve known you.

Mike: I don’t even remember how long ago that was, but it was way back.

Sonal: It might be like 15. No, not 15. Maybe 15, almost like 12 years now. I don’t know.

Mike: Could be, yeah.

Sonal: I love that you named it after yourself; I want to hear about the Masnick Impossibility Theorem!

Mike: It is impossible to do content moderation well, and there are a variety of reasons for that:

• One being that, any kind of content moderation is going to piss off someone, and that is generally the person whose content was moderated.
• The second element is that, you know, so much of this is subjective decision making, and everybody has a different view on these things… We ran a sort of conference event a few years ago, where we made everyone in the audience have to be content moderators for a number of different case studies effectively; and we had 100 content moderator experts in the audience, and none of them agreed. On every case that we did, people had strong disagreements over what should have been done about this particular content.
• And then on top of that, you just have the law of large numbers. If you’re making decisions on 500 million pieces of content a day, and you get it 99.999% correct, you’re still going to have a huge number of mistakes, however you define “mistakes.” You know, there are things that are going to be missed; there are things that are going to be taken down that probably should not have been taken down. That is going to happen, there is no way to avoid that. And in absolute numbers, because the overall set is SO large, it’s going to appear like these companies are incompetent in how they moderate content.

That is just the reality of the process of moderating content, and nothing is going to fix that. Hiring more human moderators is not going to fix that; building better AI is not going to fix that. You can improve on it but one of the nice things about Section 230 — and the way it is structured in that there is no liability for the moderation — is that it allows for different experimentation to happen.

So you have very different approaches. And, everybody focuses on Twitter, and Facebook, and YouTube — but then you have to take into account tons of other sites, including Wikipedia. Wikipedia is allowed to have all these individuals editing their platform because of 230. Or you look at another site like Reddit, right; Reddit has set up all these different subreddits, and each of them have their own moderators that allow them to set up their own rules. That’s allowed — that is possible — because of Section 230. And any of these changes could make those kinds of things impossible.

Sonal: It’s funny because in the examples you listed, you made sites that are very often used by students, like Wikipedia for research; but also, I just wanna make the point on this, that it applies to vaccine sites, and anti-vaxxer sites. It applies to all kinds of sites and that variety is partly the point here as well. I think that’s really important to underscore.

Mike: And let me underscore it even further. CDA 230 protects every website online. People say that, “Oh it’s a gift to big tech and newspapers don’t get this.” No, newspapers get it too for their website; every website gets this, and that means your personal blog. It means when you retweet someone, you get that protection as well.

All of these things, and all of these other sites, and all these other services, and everything that everyone is building — I mean lots of people listening to this are building different internet services — all of those services are protected by 230. And this matters waaaaay beyond just the big three or four companies out there.

Sonal: I am so glad you brought that up, Mike, because the most and really only alarming line in the executive order to me was this quote: “For purposes of this order, the term ‘online platform’ means any website or application that allows users to create and share content or engage in social networking or any general search engine.” And that is quite literally every site.

Mike: That is every site.

Sonal: Every site of every size. And it makes me think of the other law — it’s not Masnick’s Law of Impossibilities — it is the Law of Unintended Consequences.

And this seems true for every regulation — and I think of GDPR and all these other regulations — that all they really did, in fact, was help bigger companies, the very group they were trying not to. All the smaller players who don’t have huge compliance arms, legal officers, and the people they can hire to moderate, process queries and takedown requests get punished, which then further entrenches [them]. So it’s a vicious loop, essentially.

Mike: And that should be very scary. Because part of the executive order itself starts out by claiming that the reason they have to do this executive order is because there are limited number of social media sites out there.

And yet the definition that they have in the setup of what they’re trying to do would effectively limit that, even further, by making it impossible for new competition to show up, and for smaller sites to exist. And the more you put in place these kinds of rules and regulations, the more difficult you make it for there to be any new startups in this space, any new websites — because it becomes a costly mess for any smaller website to comply.

Sonal: Right. And while I completely agree with you that people alone or technology alone is not the answer, one thing I do want to point out about the “way forward” part of it is that this conflates the ownership of WHO decides versus the size of the company that decides.

So, for instance, instead of having like a single CEO decide, “This is my vision for this big company,” crypto is an often cited case — my partner Chris Dixon, has written an op-ed in WIRED about this a couple years ago — as a way forward for thinking about the governance of some of these sites and thinking of a crypto-decentralized native way, so that it’s “a community owned and operated service” (which is his way of thinking about it). You and I have talked about crypto many many times over the course of our friendship and years (and I think at the inaugural Copia Policy Institute, I think you had a whole section on crypto, if I remember); and I’m curious for your thoughts on that as well.

Mike: Yeah so last year, I wrote a paper for the Knight First Amendment [Institute] at Columbia University, which is called “Protocols, not Platforms.”

Sonal: Ahh, I remember this.

Mike: Oh yeah, the horcruxes.

Sonal: I teased you about it where I was like, “Mike, hallows, not horcruxes Mike!” And I myself do not love when people use Harry Potter analogies, but my god that was so perfect for that. I’m sorry. It’s very much ”Hallows, Not Horcruxes” which is great — “Protocols, Not Platforms.”

Mike: Yeah, you know, that paper discusses what the content-moderation world looks like in a distributed, decentralized system — potentially based on crypto. The paper touches on not just crypto, but just more decentralized, interoperable protocol-based systems.

And that changes a number of the content moderation questions. It doesn’t make them go away — and I do think that is one mistake that some people make, which is they think if we just set it up on a crypto-based distributed system, then we just wipe our hands of it; and it’s everybody’s individual decision, however it’s implemented, let that happen.

Sonal: It also doesn’t leave room for the variety of governance approaches that are inevitable in that as well. Because for the record, just as you’re arguing for a variety of experiments — whether it’s a privately owned, public owned company, centralized, decentralized, whichever — even in the crypto world, there’s a variety of governance approaches that can be applied, which is great. And there’s been a lot of experiments already playing out on that front when it comes to protocols.

Mike: And I think that’s good! It is that experimentation that we need.

And that experimentation is not designed just to like find the best result, but to recognize that there are different best results for different communities, and different purposes, and different services. There are certain cases where you want a Wikipedia approach; and there are certain cases where you want a Reddit approach; and there are certain cases where you want a Twitter approach; and whatever other approaches there are as well.

You can have all these different things, and some of them work [only] in some cases. The only way we’re allowed to figure that out is if we have the freedom to make those choices and see what happens.

Sonal: That’s a wonderful note to end on.

So, in this show, we ask our guests (our experts) to bottom-line it for me. And while this has been longer than 16 minutes — it’s a special long episode — bottom-line it for me, Mike. What’s the big takeaway?

Mike: So, the rules of how the internet works are under attack. This executive order by itself is not going to effectively change anything directly: It’s going to cause a lot of heat and light, but very little actual fire.

What we are seeing — and this goes beyond just this executive order — is that, people are really trying to change the way moderation works online. And we’ve already seen some laws — both in the U.S., and certainly outside the U.S. there have been a bunch of laws that are direct to that content moderation — and that is going to continue. I worry very strongly about what that does, whether that locks everyone into a specific type of content moderation, and what that means over the long term for freedom of speech on the internet.

Sonal: Thank you so much for joining this segment, Mike.

Mike: Thank you, for having me.

• 

  Mike Masnick
• 

  Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

  Follow Twitter

This week we do a short but deep dive on the opioid crisis, given the data around where and who was behind the manufacturing and distribution of specific opioids:

• How do opioids work, why these drugs?
• Who’s to blame?
• What are other directions for managing pain — and where could tech come in, even with the broader social, cultural, and structural context involved?

Our a16z experts in this episode are a16z bio general partners Jorge Conde and Vijay Pande, in conversation with host Sonal Chokshi.

Show Notes

• Historical background on the opioid crisis, and current legal cases involving manufacturers [0:00]
• How opioids interact with the brain and why they can be addictive [3:09]
• Systemic reasons that lead to excessive opioid prescriptions, and who is to blame for the crisis [6:13]
• Possible solutions for manufacturers, distributors, regulators, pharmacies, and physicians [8:30], and the need for better prescription management [13:16]
• New technologies, including VR therapeutics, that may reduce the need for opioids [15:08], and a better understanding of addiction [16:35]

Transcript

Sonal: Hi, everyone. Welcome to the “a16z Podcast.” I’m Sonal, and I’m here today with the fourth episode of our new short-form new show, “16 Minutes,” where we cover recent headlines the a16z way, offering expert takes on the trends involved and more. You can follow the show in its own feed in your favorite podcast player app.

Our other episodes cover multiple news items and topics, but this week we’re doing two separate, but short, deep dives connected to recent headlines. One on e-sports gaming and the future of entertainment, which you can find in this feed or at a16z.com/16minutes and this episode, which is on a sad but important topic, the opiate crisis.

Just to quickly sum up, the issues of the opiate crisis have been around for years, which is this prescription opioid epidemic that resulted in nearly 100,000 deaths from 2005 to 2012. And what makes it even sadder is that it disproportionately affected people from regions that are underserved economically — for instance, native American tribal regions, towns in West Virginia, and so on. For what opioids are, as a reminder, remember the word opium — they’re a class of drugs that include heroin, fentanyl, pain relievers like Oxycontin, Vicodin, Codeine, morphine, and most of those are pain relievers that are legal and available by prescription.

This crisis has been around for years, but here’s the news — the Washington Post and the publisher of the Charleston Gazette-Mail, which is a West Virginia paper — one of the regions that’s most impacted by this crisis — waged a year-long legal battle and won a court order for access to the drug enforcement administration’s database, which is this automation of reports and consolidated orders. It’s the ARCOS database. And basically, the Washington Post’s work helps visualize how much specific drugs went to individual states and counties, and who the top distributors, manufacturers, and pharmacies that were involved.

And according to the Post’s high-level findings, just three companies manufactured about 88% of the pills, and just 6 companies distributed 75% of them. And over the past couple of weeks, a number of lawsuits have been filed as a result of those findings. Arizona just filed a case against the maker of Oxycontin. Unusually, they did it directly at the Supreme court level, while towns and cities are suing pharmacies like Walmart, CVS, and Walgreens. In fact, nearly 2,000 cases have been brought as reported by the New York Times, and their headline for that story by the way, was so perfect and so starkly sad — “3,271 pill bottles, a town of 2,831.”

So, that’s a high-level summary of what’s going on, what’s in the news. I’d like to now welcome a16z Bio general partners Jorge Conde and Vijay Pande to talk about their views on this from their vantage point. Welcome, guys.

Jorge: Thank you.

Sonal: So, one bit of color from that New York Times story that is just so vivid and heartbreaking — one County in Ohio resorted to a mobile morgue just to handle all the corpses from people who died from overdoses, which is so sad. And as with all such things, science and technology does not live in a vacuum and plays out against a broader constructional context. So, I want to acknowledge that we’re going to be focusing on a specific angle, but really this is a huge problem on so many different levels. So, first of all, can you just quickly summarize the crisis from your point of view? Why opioid? What’s going on here?

How opioids interact with the body

Jorge: Well, first of all, opioids, as you said, are opium-based drugs, and it’s probably worth a moment to talk about kind of how they work and why they’re such a problem. <Yes.> Opioids basically target a receptor class within cells called the opioid receptors. And there’s three main classes, and the three main classes all have slightly different functions. And by the way, as we learn more biology — but I think identified another 15 or 20 subclasses of these things.

So, the biology, as you can imagine, is complex, but essentially what happens with an opioid is that it targets one or usually many of these receptors and that has the pain-numbing or painkilling effect. It also hits some of our, you know, essentially our pleasure-seeking centers. So, it has the addictive effect…

Sonal: Hence, the addiction.

Jorge: And by the way, it also hits other important receptors that are necessary for, sort of, our physiological function. Most notably, one of the subclasses of receptors is responsible for sending the signal to your brain that you need to breathe.

Sonal: Whoa, I had no idea.

Jorge: Yeah, no. A lot of people that overdose and die from opioids, <Oh…> really what they die from is forgetting to breathe. And in fact, like the recovery drug Naloxone, it basically competes for the drug off that receptor so the person actually comes back <Wow.> and remembers to breathe. So the drug itself is incredibly powerful, and I think one of the important things to remember is that addiction isn’t weakness. It’s not lack of willpower. It’s actually a weakness of the biology that the opioids target.

In fact, I remember when I was in graduate school, I took a pharmacology class, and the lecturer at the beginning said, you know, if I took this classroom of very accomplished, intelligent, driven, responsible graduate students, medical students, and gave everyone a dose of heroin, a significant proportion of a significant majority of this class would be hopeless addicts tomorrow.

So, a big part of the problem here is that this is a very, very powerful class of drugs. And what’s really tricky about opioids is that a more powerful drug is not necessarily a better drug.

Sonal: First of all, thank you for acknowledging that this is not necessarily a choice that people make. That’s really important, that it’s biology, but you also mentioned heroin in that example. And that one is an illegal one, which is of course a class of opioid, but most of these are prescribed. So I’m curious how that plays out.

Jorge: First of all, biology is a very dynamic system. And so if you take a drug, any drug, really, you start to — well, you tend to develop tolerance for it over time. And it can happen via various mechanisms, but one of the mechanisms that’s believed to be the case in opioids is that as you, essentially, take the drug, your receptors essentially become accustomed to it, and so it actually changes the dynamic of the receptors.

And people describe it as, you know, if you take opioids for a long time, you are quite literally changing your brain. And so the result of that is, if you’re taking a drug — and especially for relieving pain — you may need more and more of that drug to relieve pain. If that particular opioid also happens to target or hit one of the receptors associated with what’s linked to addiction, over time you’re going to seek more and more of it. So it just becomes a truly biological dependence at the cellular level for these drugs.

Origins of the crisis

Vijay: You know, it’s important to consider why patients are getting these in the first place.

Sonal: Right. Quite honestly, if that — if this is, kind of, by the biology, is that you become more addicted as you take it, why are they getting it?

Vijay: And there’s two reasons, which is somewhat of a shift. So one reason is that there’s been a recent shift in policy that essentially no pain is acceptable. So, you know, they often ask you if you’re in the ER or something, like, what’s your pain from 0 to 10. And it’s not that everyone’s saying 10 and then they get fentanyl, it’s the belief that no pain is acceptable. And this is actually very much an American thing. In other cultures, you know, you may be under extreme pain, but you’ll get tea, or you’ll get maybe Tylenol or something very different, and it’s just understood that you have to sit with the pain.

The second, man, it’s just the healthcare system now is so strained that if let’s say you have major back pain and you should maybe be seeing physical therapy or maybe you should be seeing a doctor for musculoskeletal, it may take you four weeks, six weeks to see that doctor.

Sonal: It takes time to see an expert.

Vijay: Yeah. But you could get the prescription immediately.

Sonal: So some of this is tied to health care access.

Vijay: Yeah. But then, you know, it puts them in this bind where they really should be getting physical therapy or something like that, and they are on this path. The third thing is that often the alternatives are harder short-term, like physical therapy is a lot of pain. And so this is just, it’s available, it’s thrown on you by a doctor and it’s easy. You put those things together, that’s the match on — that lights the fire.

Sonal: So, this is very helpful for helping break down the biology and the science behind this. It plays out against broader structural factors, cultural factors, political factors. This is a really big, important topic. And I have to ask, who’s to blame? Like, the interesting thing is that the news — there’s all the lawsuits happening to these pharmacies. And now the pharmacies and distributors, they’re coming back and saying, well, what about the impact of doctors and criminal drug dealers? Politicians — they are the ones who are trying to hide the database. There’s so many different players going around here. I want you guys to tell me, like, who’s to blame.

Jorge: I mean, embedded in the question is part of the answer. I think really what we have is a massive systemic failure. I mean, you talk about manufacturers, you talk about distributors, you talk about pharmacies, you talk about prescribing physicians, and ultimately, you talk about patients and their families and their caregivers and sort of the communities that support them. And then you also talk about the politicians, you know, the public health agencies.

Correcting systemic failures

I think the systemic failure here is pretty broad. So, we can start from the very beginning, which is — we do need better opioids. Now we do need better painkilling drugs. We need, as Vijay mentioned, to be more thoughtful about how and when we intervene with pharmacologic drugs for pain. One of the things that you can’t do with an opioid is you can try to design something that is only hitting the right receptor.

Sonal: This goes back to your earlier point about there being 15 types of receptors that are now being discovered. You can get more and more precise.

Jorge: Exactly. So, now that we can engineer cells and we can work with cells, we can find very precise ways to understand what molecules are interacting with what parts of the cell, and design molecules that are hitting just the right notes that we’re going to be, you know, more targeted. So, there is the potential for a better opioid. By the way, to date, most of the attempts to improve it have been to address the, you know, ways to not tamper with it, so you can’t overdose on it. But the reality is you can get a better molecule if we understand what’s driving the biology. So, that’s the first step on the manufacturing side. The second one is, yes, the distributors and the pharmacies. I mean, the biggest problem is that this is a very ad hoc, disjointed system that we have here in the United States.

Sonal: Like the healthcare system.

Jorge: The healthcare system. And so I think a lot of what you’re relying on in terms of the crisis is that there aren’t really the checks and balances and the alert systems that you would — one would expect in place that doesn’t require sort of a human being to say this, you know, employees flag one particular shipment, but that one particular shipment or that one particular prescription obviously doesn’t catch the systemic problem as it’s evolving. And so you’re really missing, you know, the forest for the tree.

Sonal: Is that a place that tech can help?

Jorge: It’s an absolute place that tech can help because, I mean, first of all, a lot of this is by requirement that you have to inform the public health agencies if there is the suspected overuse of a controlled substance. And so, instead of requiring people to voluntarily do that, you could deploy technology-based systems that essentially do that automatically.

Sonal: In fact, one of the quotes in the New York Times article came from a Walgreens official who said that he was the one who was tasked with monitoring the orders — said his department “was not equipped for that work.” I mean, that seems like an obvious place that tech could literally do what you’re describing.

Jorge: And it’s a place that tech could do it far better.

Sonal: Exactly. No, that makes great sense.

Vijay: You have to understand, I mean, what’s going on in a lot of these places, it’s Post-its, fax machines. It’s something where, you know, the things that we take for granted that on, sort of, just coordinate our daily lives — could be put in here and could really have a significant impact.

Sonal: Okay. So, let’s go back to the systemic players and failures. We have manufacturers, distributors — let’s continue breaking each one of those down.

Jorge: On the manufacturer side, there’s really two issues here. One is we do need better drugs, as we talked about. And number two — and I think this is a very important point — is, you know, a lot of times in companies as they’re commercializing drugs, obviously, the goal is to grow revenue and that can, you know, sometimes create perverse incentives to drive usage where perhaps there shouldn’t be usage. And I’m not saying that’s necessarily the case here, but that’s something that I’ve seen happen, unfortunately, across the industry over time.

The second issue is the distributors. The distributors are obviously responsible for moving product through the channel. They, of course, have incentive to move more product through the channel. And so, you know, if there are no controls in place, if the right tensions aren’t there between how things are prescribed, or how things are reordered, or how things are pulled through the system, that could also create a perverse incentive from a distributor standpoint.

And I think you show some of the concentration of what happened in the case of the — of this particular episode of the opioid crisis, as you’ve laid it out. So we do need checks against the distributors as well. When you get to the pharmacy, the pharmacy is where the rubber meets the road, right? These are where the prescriptions are getting picked up, or getting shipped to, at least. And so, if you don’t have, you know, a manual control system there — I actually think that the biggest problem is just lack of an alert system. If I go in today to pick up a prescription, there is no real system that would raise flags, at least not efficiently at the system-wide level. It tends to happen very episodically, as this story itself has shown.

And then, finally, there’s the physician prescription challenge. Because patients are in pain, the physician may not want them to tolerate pain, so may be more likely to offer this to offer immediate relief. Two, you get to the point where if you have to wait weeks and weeks and weeks to see a specialist or to get therapy, or to get treatment, this is a fast fix, short-term solution that eventually might become a longer-term problem, obviously, as addiction becomes an issue.

And the third one is — these are all related points, but physicians, for the most part, don’t have the right control systems to do really effective medication management. So my treatment of you is very episodic. I come — I see you, you describe pain, I will prescribe something. I may look in the notes and go back and see what had happened in the past, but I’m not really following this day to day.

Better prescription management

Sonal: And this by the way, applies across all health problems, not just pain and addiction. <crosstalk>

Jorge: All health problems. Medication management, medication reconciliation is a massive problem across the entire healthcare system. The particular challenge here, of course, is that this is the one area where a more powerful drug leads to more usage rather than less usage. And that’s what makes it so difficult when you can’t reconcile, you know, patient usage is happening over time.

Vijay: And there’s ways that we could work within the existing system. Like, one thing you could imagine is a PBM that is more involved…

Jorge: Pharmacy Benefit Manager.

Vijay: Yeah, Pharmacy Benefit Manager, that’s more involved with clinical care, where they’re just — they’re not the doctors, but at least they’re better interfacing with the doctors such that you can at least have sanity checks. Like, there’s no reason why a patient would need this. And this way you can’t shop around to multiple pharmacies because you’ve got the same PBM, and it is that layer.

And I think as you start to get smarter PBMs, these problems would be very naturally addressed. Not just for the opioid crisis, but it would be true for patients that have sometimes two or three drugs to treat the same condition, or three drugs that are actually gonna interfere with each other. Those are sometimes very difficult because in the medical system, you’ve got the endocrinologist and the cardiologist and the psychiatrist each prescribing without really any coordination.

Jorge: And you know, to that exact point, we have a problem in the healthcare system of getting things de-prescribed.

Sonal: What do you mean by that, de-prescribed?

Jorge: Well, patients might be taking a medication for an acute condition. And, you know, I saw the physician and the physician told me to take this medicine for condition X…

Sonal: You got a — you broke your arm and you need Vicodin…

Jorge: Or you may have, you know, you may have a heart condition that has a — that’s going through an acute episode. Any number of things that I’m on 10 different medications, it could be that the condition for which this one drug was given to me has since been alleviated, has since been addressed…

Sonal: But that information doesn’t get plugged back into the system to close that loop…

Jorge: Yeah, and I don’t know how to stop taking it, so I might be taking a medication that I don’t need for a long period of time. And if somebody doesn’t do the reconciliation that Vijay just described, I could be on many medications that not only interfere with each other, which is a problem, but that I may not even need, which is a different problem.

Possible tech solutions

Sonal: So, that kind of addresses it at the, sort of, structural, logistical level of the healthcare system. Now, back to the point you brought up about the biology and some of the pain management. I mean, there’s obviously alternatives like TENS devices, and all kinds of things that could potentially scale in the future to address pain, but now let’s go to what the fixes are. Obviously, there’s social societal things that need to be addressed, but what can tech and science help with here? Are there any other future directions from your vantage point on the bio side? Clearly, there’s technology to address the transparency, the PBMs, the pharmacy Benefit Managers, closing the loop, everything from manufacturer distribution to prescription. What are some of the other things, what are some of the interesting directions you see to help address this?

Jorge: Well, there are efforts to develop digital therapeutics, VR-type applications…

Sonal: Right. And by the way, digital therapeutics as in, like, apps and things like technology that can actually be — act as if a drug in helping people to better outcomes?

Jorge: Exactly. Maybe that can help you — get you into a state of mind that might help alleviate the pain, right? So, you know, if you can find different ways to address the pain issue, whether it’s, you know, physical therapies or something maybe novel like, you know, quite literally having a VR, virtual reality-type experience, or having an application on your phone that helps you meditate or calm down that might address some of the pain issues, you may not be as dependent on getting on the opioids in the first place.

Sonal: I’ve read a ton of papers, actually, that VR has already proven to be effective in helping with PTSD — post-traumatic stress disorders — with veterans coming back from wars, or, you know, people who are suffering severe depression. It’s just really amazing that it can help.

Societal influences on addiction

Vijay: Well, you know, I think often we are worrying about the consequences without thinking about the source. Jorge made a great point about how addiction is a natural consequence. There are other recent studies that talk about, sort of, a little deeper about why this is so.

So, the famous one is called the “Rat Park” study, where they actually had rats in a cage, which is kind of like jail, and given the choice between food or opioid, they’ll take the opioid until eventually, they kill themselves. But if you give them access to Rat Park where they can play and be social and, sort of, just live their normal happy lives, then actually giving the same choice they would choose the food and not the opioid.

We know that social determinants are a key part of healthcare — it’s just not wrapped into a fee-for-service kind of system, where no one’s job is to take care of these things. But if we could take care of the root causes of this, which are beyond just about prescribing drugs but about thinking about healthcare as a societal issue, I think then we can actually really have a huge effort.

Jorge: And there are several efforts ongoing now to use technology to help try to pull in all of those stakeholders in the community that can have such a big impact on some of these social determinants of health. Without that, this is another example of a fragmented system. A very analog system is — you’re doing this with call sheets, and coming up with referral names, and calling and trying to get appointments, and, you know, inbound visits and things like that.

And it’s all necessary because this requires human intervention, but the coordination shouldn’t also be human. So I think technology has an opportunity here to have a massive impact on how we coordinate all of these stakeholders. So the people that may be more susceptible given some of these social determinants are more supported.

Sonal: Right. And it just goes back to the bottom line for me, though, which is [that] technology is social, and it lives in a broader cultural context that clearly plays. Well, thank you so much, Jorge and Vijay, for joining the a16z podcast “16 Minutes.”

Vijay: Thank you.

Jorge: Thanks for having us.

• 

  Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

  Follow Twitter
• 

  Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

  Follow Twitter
• 

  Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

  Follow Twitter