How Spotify Achieved a Voluntary 99% Internal Platform Adoption Rate
“There was this first revolution, where everything became a microservice, and that kind of set us free in certain ways. And then there was the Kubernetes revolution that just shaped the cloud native landscape. And as cloud computing just gained momentum, there was this need [for] efficient orchestration and management across a diverse set of environments and whatnot. All of this, brought a lot of agility for teams to experiment,” Helen Greul, head of engineering for Backstage at Spotify, told The New Stack.
“But the downside — there has to be a downside — in our case, was the growing burden on developers. I’ve seen this with my own eyes, this developer mandate expanding,” she continued, reflecting on the impetus for the seemingly sudden rise of platform engineering.
Platform engineering is the sociotechnical mindset and set of best practices, toolchains and workflows that look to reduce friction and increase developer productivity. This year, when everyone is trying to do more with less, has seen a meteoritic rise in popularity of this discipline. We’ve also witnessed the exponential increase in adoption of the most popular open source internal developer platform or IDP: Backstage.
It was only natural that The New Stack sat down with Spotify to learn about why they invested so much time and money in Backstage, which they open sourced for any team to leverage — and how they got their own developers on board to happily use it, too.
The Origin of Backstage at Spotify
“As a hiring manager, you would first hire an engineer that would code, but then they would [have to] code and ship. Then they would need to care about runtime. Then all of a sudden, they need to manage across environments and then optimize,” Greul said, of how the developer mandate continued to explode and expand at Spotify.
Around five years ago, Spotify engineering realized they could no longer face complexity by just hiring more developers. And the modern concept of platform engineering was born, with the goal of streamlining software development.
“We have a platform for platforms here at Spotify, that’s how obsessed we are.” Greul remarked that Backstage is the biggest example, but the popular audio streaming service has several other internal ones, all at least inner-sourced.
“I think if you’re building a product for developers, it’s either going to become a platform — if it’s successful enough, if it lives long enough — or it’s just going to die out.” — Helen Greul, Spotify
Some would say the 2023 rise of platform engineering is a sign that the pendulum is swinging back from developer autonomy toward platform-driven command and control. Not Greul. She thinks the platform is the best way to deliver developers choices, giving them the ability to build onto those golden paths that they want.
“Developers love their platforms and products to be extensible,” she said. “There are very few turnkey solutions that I can think about that are used by developers and it’s true for almost all software that we use.”
Getting Started with Platform Engineering at Spotify
Spotify may stand out at the forefront of company culture and innovation, but its first goals ring true for most companies starting on their platform engineering journey. First and foremost, they wanted a catalog to create clarity of ownership.
“It was hard to know who owned the majority of microservices — there was a sprawl,” Greul said. The concept of ownership was really important to us,” so this internal developer platform kicked off with a software catalog — just components, teams, and a few other abstractions, she outlined. This became the first slice that morphed into Backstage as we know it today.
Next — again, not unusual — Spotify engineering looked to abstract the complexity of Kubernetes. The nascent platform team was looking to address the problems of orchestration, management, deployment, testing, and health tech quality.
“And that was the point where it exploded internally. When we opened up,” Greul said.
The next demand was for a single pane of glass to understand all the services eventually housed in the internal developer platform, with inventory and a release launchpad.
Also Read: Metrics-Driven Developer Productivity Engineering at Spotify
Inner Source Your Platform Engineering Experience
If they build it, they will use it.
Greul attributes Spotify’s successful platform strategy to its strong culture of inner sourcing: “Everyone is free to contribute to any repository that’s available within the company. There is no codebase that is locked.”
Long before this new age of internal developer platforms and the fashion of a focus on developer experience, inner sourcing was a companywide policy from early on at Spotify, as a way to ensure organizational fluidity. After all, she remarked, the launch of an audiobook can leverage as many as 50 different repositories. Everyone should have the chance to improve their own developer experience. Bonus: more people building it means more people eager to use it over other options — and evangelize to their colleagues about it.
Even at Spotify, Backstage usage is not mandatory. “We try to work on people’s incentives and make it easier to do the right thing,” Greul continued. “And the right thing is a bit of standardization, not too much, not too little, but just enough.”
Still, at the home of Backstage, platform engineering adoption is high. But she says that’s because Spotify squads adhere to a philosophy that standards can set you free. Spotify’s version of Backstage includes three choices of frontend technology and three of backend, in what they call their technology radar.
This tool choice is reevaluated frequently, including in quarterly surveys for measuring developer productivity. They offer great incentives to stay on the golden paths including widely promoting wins and sharing survey results internally.
The New Stack has already written about how these golden paths have been proven to cut Spotify’s developer onboarding time down from 60 days to 20.
“As a new joiner, joining Spotify, you’re kind of almost being handheld through the journey with those standards and boilerplate templates in mind,” Greul said. “You have a bit of room [to] pick one or the other, but there are some guardrails in place that are part of your onboarding.”
Of course, new tooling may arise within the organization, but she assured that the plug-and-play style of Backstage allows that best practices can be integrated fairly easily.
In the end, after about a year of internal onboarding and promotion, she reckons 99% of Spotify’s development team had adopted Backstage — because it was convenient to do so. She points to their incentives or Platform as a Product mindset to why that rate is so high.
Also Read: How Spotify Adopted and Outsourced Its Platform Mindset
Why Is Backstage Adoption Rate Stagnant at 10%?
Spotify is famously open source by default, which Greul postulates is because “if you want something to become [an] industry standard, there is no way software can be proprietary,” arguing that all technical standardization, at least over the last decade, has been via an open source pathway. This is how Spotify is paving so-called golden paths at more than 260 organizations.
But Spotify isn’t your average company culture. Greul admitted that many other companies that have adopted Backstage for platform engineering are stuck trying to get past the 10% adoption rate.
“Oftentimes it happens that they hit a road bump, or maybe the rollout is not as easy as they would have hoped,” she said. “This is where sometimes the adoption kind of struggles or stagnates or it doesn’t go beyond the POC [proof of concept.]”
It all comes back to the incentives, she said, “Developers have to see how this is beneficial for their day-to-day.” For a startup, their cognitive load could be just fine with the AWS Management Console, and not be motivated to change. “But once you reach a certain scale, it sort of becomes almost a necessity to have the tools to combat the cognitive load.” Usually, an IDP becomes necessary.
You don’t just have to sell the developers on it either. Business stakeholders need to understand that an investment in platform engineering is necessary to improve developer productivity.
Just remember that Backstage isn’t an out-of-the-box panacea. In fact, Greul said that Backstage could be a bad fit for monolithic architecture.
But, consistently across interviews, we’ve heard that the number one thing developers want from their internal developer platform is extensibility — platform engineering is not about top-down, command-and-control platforms.
“The unique kind of selling point of Backstage is that you can make it look and feel your own. You can make it so you can customize it to solve your specific needs,” Greul explained. “For developers, there is no usually one-size-fits-all solution and they love that they can keep their unique kind of culture, unique set of tools, and unique flavors that make it look and feel like it’s a homegrown solution, even though it’s not.”
Most importantly, it has to solve their problems. Their first concerns are usually discoverability and standardization.
Of course, she remarked, this journey of discovery doesn’t start with deciding you need an internal developer portal. It’s about clarifying what are the challenges to developer productivity, what are keeping developers stressed and up at night.
In the end, it’s as Greul said, “With a product that’s built for developers, with empathy for developers, they recognize the value, and it’s just easier for them to use what’s already available than spinning something weird on the side.”
