Spotify dances to the open source beat

Let the OSS Enterprise newsletter guide your open source journey! Sign up here.

Just about every technology company under the sun wants to align themselves with the open source, whether it’s Facebook open sourcing its own internal projects or Microsoft doling out north of $7 billion to acquire one of the biggest platforms for open source developers — GitHub.

Spotify is no different. The music-streaming giant has open-sourced a number of its projects through the years, such as Backstage, which was recently accepted as an incubating project at the Cloud Native Computing Foundation (CNCF) after two years as an open source project. The company also recently joined the Open Source Security Foundation, opened a dedicated open source program office, and is now launching a fund to support independent open source projects.

In short, Spotify is doubling down on its open source efforts.

Open for business

There are many reasons why a company might choose to open source its internal technologies, or contribute to those maintained by other companies or individuals. For starters, it can help engage the broader software development community and serves as a useful recruitment tool. A company may also contribute resources to community-driven projects where it plays a central part of their critical infrastructure, to help bolster security, for example.

Backstage, for its part, is all about building customized “developer portals,” unifying a company’s myriad tooling, services, apps, data, and documents in a single interface through which they can access their cloud providers’ console, troubleshoot Kubernetes, and find all the documentation they need as part of their day-to-day work.

“The problem Backstage solves is complexity — the kind of everyday complexity that can really bog engineers and their teams down, which then slows your whole organization down,” Tyson Singer, Spotify’s head of technology and platforms, told VentureBeat. “Backstage as a product and as a platform is really about creating a better experience for engineers — streamlining their workflows, making it easier to share knowledge, and getting the messy parts of infrastructure out of their way. It enables them to better focus on building business value — innovative products and features.”

Today, Backstage is used by dozens of companies, spanning retail, gaming, finance, transport, and more, including Netflix, American Airlines, IKEA, Splunk, HP, Expedia, and Peleton. But when all is said and done, what does Spotify get from open-sourcing Backstage? Well for starters, it gets a better version of Backstage for itself due to the community-driven nature of the project.

“Let’s imagine the counterfactual, where two years ago we didn’t open source Backstage, and instead we poured the same amount of internal resources into it as we have gotten from the external community — and based on the tremendous community engagement so far, that would have been a huge investment and tricky to fund — it still would not be as good a product as it is today,” Singer explained. “A diversity of viewpoints and use-cases, from adopting companies like the world’s biggest airline or fast-growing finance startup, individual contributors and third-party software providers, has improved the product, making it more robust and enabling the platform to keep up with the pace of change going on both inside and outside a particular company.”

But on top of that, the fact that Backstage is seeing adoption at some of the world’s biggest companies indirectly benefits Spotify too, insofar as it ensures that its own product is among the de facto “developer portal” tools.

“If we had not open-sourced [Backstage], we’d be the only ones using and depending on Backstage,” Singer continued. “If eventually a different open source solution emerged, we would have had to migrate to that solution, as the community-fed innovation eclipsed our ability to keep pace.”

To support its ongoing efforts in the open source realm, Spotify has joined a long legion of companies to launch a dedicated open source program office (OSPO), designed to bring formality and order to all their open source efforts, align OSS project goals with key business objectives, manage license and compliance issues, and more.

Spooling up

Spotify has, in fact, had an OSPO of sorts for the better part of a decade already, but it constituted more of an informal group of employees who had other full-time roles at the company. Moving forward, the company now has a full-time OSPO lead in Per Ploug and is actively hiring for other roles.

So up until now, Spotify’s open source work has been driven chiefly by the “passion and engagement” of the company’s engineering teams, according to Singer.

“The enthusiasm has always been there, and we just needed to channel it,” Singer said. “A dedicated OSPO brings more clarity to this process for everyone, including what expectations are, and what kind of support should be expected. It ensures that our efforts are properly prioritized and integrated into the way we work. We want to treat it [open source] with the same level of ownership and dedication as we do with our internal applications — creating a formal OSPO allows us to do that.”

Spotify’s OSPO is positioned within the company’s “platform strategy” unit — however, it will ultimately straddle multiple teams and departments given that open source software intersects with everyone from engineering and security, to legal, HR, and beyond.

“Engineering teams have their areas of expertise — but we want our OSPO to go wide across multiple teams,” Singer said. “The best position to do that is from within our ‘platform strategy’ organization, which is the connective tissue between various R&D teams. It gives the OSPO visibility and independent positioning within that framework. It very well represents how intertwined open source is with ways of working not only in Spotify, but actually in any modern technology company.”

A central component of any OSPO is security — ensuring that any open source element in the company’s tech stack is safe is kept up-to-date with the latest version, and also compliant with the terms of the open source license. So it’s perhaps timely that Spotify recently joined the Open Source Security Foundation (OpenSSF), a pan-industry initiative launched by the Linux Foundation nearly two years ago to bolster the software supply chain.

With incumbent members such as Google, Microsoft, and JPMorgan Chase, Spotify is in good company, and its decision to join followed the critical Log4j security bug that came to light late last year. The OpenSSF also highlights how open source has emerged as the de facto model for cross-company collaboration — everyone benefits from more secure software, so it makes sense if everyone pitches in together.

“Open source security is a topic that affects every tech company — or, really, any company that relies on software,” Singer said. “We all depend on the open source ecosystem, which is why as a technical community we all have a responsibility to improve security where possible. As when we joined others in creating the Mobile Native Foundation, we see the problem as one of scale — how do you create solutions that can affect, not just local problems, but an entire landscape? We believe that participating in foundations — working together with other big companies who think about the problems and opportunities of scale within their own businesses every day — makes a lot of sense for finding scalable solutions.”

Show me the money

To further align itself with the open source realm, Spotify today lifted the lid on a new fund for “independent” (i.e. not Kubernetes) open source project maintainers. The Spotify FOSS Fund will start out at €100,000 ($109,000 USD), with the company’s engineers selecting projects they feel are most deserving of the funds, and a separate committee making the final decision. The first tranche of chosen projects will be announced some time in May.

“The idea for Spotify’s FOSS Fund came about by asking ourselves, what could we do to help support the quality of open source code that we all depend upon?,” Singer said. “It’s only natural for the larger tech players to play a role in supporting the open source ecosystem. We use it, we contribute to it, we’re building projects for others to contribute to and depend upon — we feel it’s important and necessary for us to contribute to the success of this community.”

However, €100,000 isn’t a huge amount of money on the grand scheme of things. Over the past year, we’ve seen Google pledge $100 million to support foundations such as OpenSSF and commit $1 million to a Linux Foundation open source security program. Recently, Google also partnered with Microsoft to fund another security program called the Alpha-Omega Project to the initial tune of $5 million.

But it’s perhaps unfair to compare supporting foundations and larger projects with smaller-scale “indie” projects that receive no financial backing whatsoever. Plus, it is still early days for the Spotify FOSS Fund, and it’s likely it will evolve over time — which could mean a bigger pot.

“The fund will start with €100,000 — the keyword being ‘start’,” Singer explained. “We’re ready and willing to grow the fund, but we’re using this initial amount to help us evaluate what kind of impact we can make. Funds will be distributed to ensure the maintainers have the financial means to continue maintaining their projects, fix security vulnerabilities, and continue improving the codebase. We will target projects that are independent, actively maintained, and relevant to our work here at Spotify.”