🚨 How Twitter’s child porn problem ruined its plans for an OnlyFans competitor

Zoe Schiffer and I reported and wrote this story, which appears in its entirety at The Verge.

In the spring of 2022, Twitter considered making a radical change to the platform. After years of quietly allowing adult content on the service, the company would monetize it. The proposal: give adult content creators the ability to begin selling OnlyFans-style paid subscriptions, with Twitter keeping a share of the revenue.

Had the project been approved, Twitter would have risked a massive backlash from advertisers, who generate the vast majority of the company’s revenues. But the service could have generated more than enough to compensate for losses. OnlyFans, the most popular by far of the adult creator sites, is projecting $2.5 billion in revenue this year — about half of Twitter’s 2021 revenue — and is already a profitable company. 

Some executives thought Twitter could easily begin capturing a share of that money since the service is already the primary marketing channel for most OnlyFans creators. And so resources were pushed to a new project called ACM: Adult Content Monetization.

Before the final go-ahead to launch, though, Twitter convened 84 employees to form what it called a “Red Team.” The goal was “to pressure-test the decision to allow adult creators to monetize on the platform, by specifically focusing on what it would look like for Twitter to do this safely and responsibly,” according to documents obtained by The Verge and interviews with current and former Twitter employees.

What the Red Team discovered derailed the project: Twitter could not safely allow adult creators to sell subscriptions because the company was not — and still is not — effectively policing harmful sexual content on the platform.

“Twitter cannot accurately detect child sexual exploitation and non-consensual nudity at scale,” the Red Team concluded in April 2022. The company also lacked tools to verify that creators and consumers of adult content were of legal age, the team found. As a result, in May — weeks after Elon Musk agreed to purchase the company for $44 billion — the company delayed the project indefinitely. If Twitter couldn’t consistently remove child sexual exploitative content on the platform today, how would it even begin to monetize porn? 

Launching ACM would worsen the problem, the team found. Allowing creators to begin putting their content behind a paywall would mean that even more illegal material would make its way to Twitter — and more of it would slip out of view. Twitter had few effective tools available to find it.

Taking the Red Team report seriously, leadership decided it would not launch Adult Content Monetization until Twitter put more health and safety measures in place.

The Red Team report “was part of a discussion, which ultimately led us to pause the workstream for the right reasons,” said Twitter spokeswoman Katie Rosborough.

But that did little to change the problem at hand — one that employees from across the company have been warning about for over a year. According to interviews with current and former staffers, as well as 58 pages of internal documents obtained by The Verge, Twitter still has a problem with content that sexually exploits children. Executives are apparently well-informed about the issue, and the company is doing little to fix it.

“Twitter has zero tolerance for child sexual exploitation,” Twitter’s Rosborough said. “We aggressively fight online child sexual abuse and have invested significantly in technology and tools to enforce our policy. Our dedicated teams work to stay ahead of bad-faith actors and to help ensure we’re protecting minors from harm — both on and offline.”

Read the rest of this story at The Verge.

Bonus links

• Elon Musk amended his lawsuit to get out of purchasing Twitter to include the new allegations from the company’s former head of security, Peiter “Mudge” Zatko. (Giles Turner / Bloomberg)
• Twitter rolled out Circle, its take on Instagram’s Close Friends feature, around the world. (Ivan Mehta / TechCrunch)
• The California Senate passed an online safety bill for children requiring apps and websites to take proactive steps to protect children, creating a new tangle of regulations that critics say could introduce new harms. (Natasha Singer / New York Times)
• After Mark Zuckerberg’s appearance on Joe Rogan’s podcast, where he discussed a decision to limit the reach of stories about Hunter Biden’s laptop, the FBI issued a statement saying it only warns companies but does not require them to take action. (Dennis Romero and Ben Goggin / NBC)
• Facebook is ending support for its standalone Facebook Gaming app and bringing some of its features to the main app. (Jordan Middler / Video Games Chronicle)
• Google hasn’t approved Truth Social’s Android app because of its insufficient content moderation. (Sara Fischer / Axios)
• Microsoft said it would relax policies that made it prohibitively expensive to run Windows and Office workloads on competitors to its Azure platform after European Union antitrust regulators began looking into it. (Paul Thurott)
• Apple said paid podcast subscribers have tripled since June 2021, though it wouldn’t provide real numbers. (Sara Guaglione / Digiday)
• Apple appears to be considering “Reality One,” “Reality Pro” and “Reality Processor” as trademarks for its forthcoming mixed-reality headset. (Mark Gurman / Bloomberg)
• Snapchat added Dual Camera, which lets people take photos and videos from the front and rear cameras simultaneously. (Karissa Bell / Engadget)
• Y Combinator named Garry Tan, co-founder of Initialized Capital and longtime partner to the startup accelerator, as its next president. (Alex Konrad / Forbes)

Those good tweets

Talk to me

Send me tips, comments, questions, and red team documents: casey@platformer.news.