Spotify’s Backstage Roadmap Aims to Speed Adoption
There are now about 2,200 companies who’ve adopted Backstage, quadruple where it was at the start of the year. In many ways, the state of platform engineering in 2023 has been tied to this open source framework for developer portals, donated to the Cloud Native Computing Foundation by Spotify back in 2020.
Early adopters outside of Spotify are sharing their success cases too, including at Expedia, Toyota and Shutterstock. But not all companies have found it so easy to adopt Backstage across their engineering teams. But that may change over the next couple of months.
Ahead of today’s webinar on the Spotify for Backstage Roadmap, The New Stack sat down for an exclusive interview with Meg Watson, head of product at Backstage at Spotify, to learn where the open source project is heading, and how it should soon become easier to adopt and maintain.
Easier Onboarding and Maintaining
Backstage was born out of a necessity at Spotify, the media streaming company.
As the cloud native organization scaled, Spotify engineering realized it could no longer support full developer team autonomy and it was time to put up some guardrails. The streaming service settled on the choice of three frontend and two backend services that the platform engineering team supports. Following the belief that “standards can set you free,” when Spotify laid its own golden path, it then open sourced Backstage to become an industry standard.
But, from the start, Spotify engineering acknowledged that no organization — especially one built on the complex cloud native landscape — is the same. So instead of building a one-size-fits-all internal developer portal (IDP), the team built a platform that then enables teams to build their own custom portals for their own custom needs.
However, with too many choices and too much extensibility, Backstage, while still the most popular open source IDP, has experienced various stages of adoption externally. In fact, according to Helen Greul, head of engineering for Backstage at Spotify, the average Backstage adoption rate is stuck at 10%. This is often because adopters don’t make it past the proof of concept, she told The New Stack, because they fail to understand the challenges developers are facing and build the thinnest viable platform around solving those first.
Slower adoption can also be due to simply too many steps and choices — more than 70 steps and 100 plug-ins! — making it overwhelming to get started.
That’s why the Spotify team is soon releasing features that enable faster adoption and simpler maintenance of an IDP on Backstage. And for users who still want a deeper level of customization, they can still fork the project at a lower level of abstraction.
Backstage Quickstart
This November, Spotify will release into private beta a no-code setup solution called Quickstart that condenses those more than 70 getting-started steps into just four or five. This “is going to fundamentally make it a lot easier to get started with your Backstage instance,” Watson told The New Stack, and then “reduce the maintenance overhead associated with keeping your Backstage instance up to date.”
Quickstart is a visually driven user interface (UI) that leverages a configuration manager or directs you to a marketplace (also soon to be released).
“You no longer need to worry about managing a codebase alongside Backstage, so it’s far simpler to get started and to stay started with Backstage,” Watson said. “One of the big pain points that Quickstart will address is that we are able to provide adopters with essentially a golden path to follow to get started with Backstage,” especially getting started and a broad do-it-yourself approach isn’t ideal.
“Instead we’ll have some guidance to help them stay on the path and make sure that they’re well supported, particularly in the early stages of their journey.”
Declarative Integration for Backstage
When this new declarative integration feature is enabled, Backstage will automatically detect any installed plug-ins and add them to your Backstage application, which is by default shipped with what’s been identified by the plug-in owners as the preferred configuration settings.
This declarative integration should not only help teams onboard Backstage more easily, but also to continue to bake in new features.
A Backstage Plug-In Marketplace
Once Backstage became an open source project, it was opened up to an open source ecosystem that included other tools, such as those offered by Spotify and its partners. Like all platform engineering themes, we know that too many choices can become simply too many. That became the case with the plethora of plug-ins to add on and build on top of.
“We’ve heard loud and clear from our adopters that, while we have a really robust set of capabilities inside the Backstage ecosystem, with more than 100 plug-ins, it can be difficult to understand which ones are secure, well-maintained and essentially supported,” Watson said.
In November, Spotify will launch a new marketplace for Backstage plug-ins, which includes some — from Red Hat, VMware, CircleCI and PagerDuty — marked as by Spotify trusted partners, which have proven commitment to maintaining their plugin projects. They are also looking at ways to further measure the security of the various plug-ins, along with other potential attributes.
“We just want to make sure we have a really robust set of solutions that folks can use as their needs evolve,” Watson said. “What we hear is that, oftentimes, as an organization scales or gets more mature, the complexity only increases. So it’ll be ever important for adopters and customers to be able to go to a single marketplace and add whatever they need across the full lifecycle of using Backstage.”
New Backstage Commercial Offerings Coming Soon
On top of the open source offering, Spotify has an enterprise offering, including support, as well as five different plug-ins, which looks to further proliferate the lessons the company has learned from the greater Backstage user base.
These plug-ins include Soundcheck, a no-code UI for tech health checks, which has proven to decrease main code build failures by 20% and high urgency incidents by 25%, and Skill Exchange, an internal learning marketplace with a Slack integration, which is directly tied to a jump in retention and internal promotions.
Following this success, Spotify will soon release three more commercial plug-ins to its commercial plug-in bundle:
- Data Experience, which will allow organizations to view all their data directly within Backstage.
- Cost Insights, a FinOps and GreenOps tool that will enable teams to see their cloud costs from within Backstage and provide recommended actions.
- FleetShift, which will enable automated code changes to fuel your team’s productivity and improve the security of products and applications, including carrying out code migrations, resolving security incidents, and automating version management.
Beyond these, Spotify is also building Confidence, an experimentation platform that allows for A/B testing of platform features and more. Confidence is available to be considered via demo, and is currently in closed beta.
“At Spotify, we see frequent Backstage usage means that folks are more productive [and] they are shipping better code,” Watson said. “We’re really excited to be able to share these learnings with the community broadly.”
Any teams interested in participating in any of these private betas are welcome to sign up via Spotify’s interest form.
