A few words about FiveThirtyEight

Silver Bulletin will begin publishing some new features soon — including Trump approval ratings. But the media just lost an extremely valuable resource.

Last night, as President Trump delivered his State of the Union address¹, the Wall Street Journal reported that ABC News would lay off the remaining staff at 538 as part of broader cuts within corporate parent Disney. Having been through several rounds of this before, including two years ago when the staff was cut by more than half and my tenure expired too², I know it’s a brutal process for everyone involved. It’s also tough being in a business while having a constant anvil over your head, as we had in pretty much every odd-numbered (non-election) year from 2017 onward at 538/FiveThirtyEight.³ I don’t know all of the staffers from the most recent iteration of the site, but the ones I have met or who I overlapped with are all extremely conscientious and hard-working people and were often forced to work double-duty as jobs were cut but frequently not replaced. My heart goes out to them, and I’m happy to provide recommendations for people I worked with there.

Beyond that, I wasn’t inclined to say too much more, but it felt weirder not to say anything at all. And it’s easier to say something here than filter it through a reporter or something.

For more extended thoughts on the environment at Disney — plus plenty of self-reflection/self-criticism — you can see the item at the bottom of SBSQ #12.⁴ But the basic issue is that Disney was never particularly interested in running FiveThirtyEight as a business, even though I think it could have been a good business. Although they were generous in maintaining the site for so long and almost never interfered in our editorial process, the sort of muscle memory a media property builds early in its tenure tends to stick. We had an incredibly talented editorial staff, but we never had enough “product” people or strategy people to help the business grow and sustain itself. It’s always an uphill battle under those conditions, particularly when it comes to recruiting and retaining staff, who were constantly being poached by outlets like the New York Times and the Washington Post.

The FiveThirtyEight.com homepage as of its relaunch under ESPN/Disney on March 17, 2014.

“Data journalism” has a bad name but a bright future

It also doesn’t quite feel like the end, exactly. “Data journalism” may have been a dumb name for what we were doing — that one’s on me — and Fivey Fox aside, the FiveThirtyEight brand was never warm and cuddly. But it always found a huge audience, and coverage of polls and political data is now much smarter. Compare the extremely analytical polling deep dives that Nate Cohn is doing at the New York Times, for instance, to the vibes-based coverage of the Boys on the Bus era. That trend may get even more entrenched as former 538ers form a diaspora that filters out to the rest of the media.

Also, I’d like to think we’re carrying a piece of the FiveThirtyEight torch here at Silver Bulletin. So, just two more quick beats. One about our near-term plans here — in slightly awkward timing, we’re launching our Trump approval ratings dashboard tomorrow. But first, a shout-out to one of the extremely valuable functions that FiveThirtyEight provided.

Collecting and maintaining a database of public polls is a lot of work, requiring diligence, meticulousness, and dealing with constant complaints about edge cases from readers and pollsters. But it’s also a public service. Polling has its challenges, but I believe it’s vital in a democracy. People only get to vote every two to four years — if they’re lucky enough to live in a state where their vote even matters. While being too “poll-driven” has pitfalls, the alternative isn’t necessarily enlightened governance. Rather, left to their own devices, elected officials are often inclined to follow some combination of (i) narrow self-interest, (ii) the loudest voices in the room, and (iii) elite opinion, which often doesn’t match broader public opinion.

FiveThirtyEight had long made its data publicly available through APIs and other tools. At Silver Bulletin, I hope we’re upholding that tradition too — although admittedly with a twist. For instance, not only are our pollster ratings (which apply the methodology I originally developed for FiveThirtyEight⁵) publicly available, but so is the underlying database of more than 12,000 polls that populates them. And for our presidential election forecast last year, our polling averages were free for everyone — though, here’s the twist, of course — the probabilities that the model spat out were paywalled. Even so, there were literally dozens of data visualizations and downloadable files just beyond that paywall — we’re not giving away the cow (the model code), but you’re getting basically everything else.

I think there are really only two options to provide an adequate incentive for the hard work of keeping a comprehensive public polling database:

• Either you need to have an upsell — most of the data is free, but premium features and/or models require a subscription; that’s basically what we’re doing.

• Or you need to have some sort of not-for-profit structure. If a university or nonpartisan nonprofit out there wants to maintain a consortium of public polls, I’m happy to have those conversations and contribute ideas (and possibly data). Although there are some benefits from duplication — everyone finds some polls that others miss, and everyone has slightly different standards — there are also diminishing returns to this.

For last year’s election, 538’s publicly-available databases were a major data source for the NYT and other news outlets that reported on the election or crunched the numbers, including Silver Bulletin. Basically, our process was to ingest their data and then make various additions, subtractions, and changes. On balance, we were slightly stricter than 538 about what polls we considered to meet a “scientific” standard — but there were edge cases that ran in both directions. (Eli is also sort of a wizard at finding polls faster than other people do.)

A bit more about our plans at Silver Bulletin

Going forward, we will continue to ensure that the database of polls that feeds into our models is publicly available, even if the outputs of the models aren’t. This was a bit superfluous before just because of the heavy overlap with 538 — indeed, we used nearly the same database structure since it’s the one I helped to design during my time there. But it’s less so now. So we’ll work to improve the discoverability and formatting of this data too when we’re publishing models and landing pages.

That doesn’t necessarily mean we want to be the “polling aggregator of record” for the entire industry, as FiveThirtyEight and Pollster.com formerly were. We have Trump approval ratings launching tomorrow, and we’ll probably release our version of generic congressional ballot averages beginning late this year. Eli and I can probably handle those two categories on our own. The midterms next year could be more of a challenge, with the abundance of House, Senate, and gubernatorial polls. But the solution is simple: we’ll hire additional staff next year as needed.

There are also some less sexy things that don’t fall into these buckets, however. For instance, vice presidential approval ratings, polls for special elections, and favorability ratings for other public figures. We’re very tempted by the idea of running an Elon Musk favorability tracker, but otherwise, these aren’t part of our plans in the short run. However, just as I don’t want to make any commitments, I also don’t want to rule anything out.

In the short term, though, we have some pressing plans:

• Trump approval ratings go up tomorrow and will be updated several times a week going forward.

• Our new SBCB men’s college basketball ratings are also ready except for finalizing the charts, and will begin publishing on Friday or Saturday.

• Then Selection Sunday is March 16, and within a day or two, we’ll also have our March Madness projections up for both the men’s and women’s brackets.

• Meanwhile, I’m hiring an Assistant Sports Analyst with an eye toward getting NFL and NBA models ready for next season.

It’s sort of a quirk that all of this is happening at once. For now, we’re just two people, and I had to get through some travel and other bottlenecks. But we’re trying to take a sustainable trajectory here, even if it’s slow. We greatly appreciate your support and hope you’ll stay with us for the long haul.

1. Technically, what most people think of a State of the Union is an “Address to Joint Session of Congress” during the first year of a president’s term. But it’s the same thing for all intents and purposes.

2. I wasn’t technically laid off — but my existing contract was set to expire in June 2023, and there was profound mutual disinterest in negotiating a new deal.

3. FiveThirtyEight was the original brand name; more recently, Disney began stylizing it as “538”. I’ve alternated between these usages here based on the era I’m writing about.

4. This was originally paywalled but it’s now been permanently unlocked.

5. I retained much of the pre-2023 FiveThirtyEight IP and models, though not the brand name, as part of the deal I originally signed with Disney in 2013.