Written by Sumeet Bansal
on 

Data-Driven Music Curation

The history of music listening is the history of culture. Back when radio was how most people discovered new music, radio DJs were the gatekeepers. Whatever they chose to play would reach millions of ears, and record labels would pay handsomely for spins.

KSDT UC San Diego’s own college radio station (KSDT).

Then, of course, came portable listening devices and finally the internet and the origins of streaming with Napster and iTunes. Now we have plenty of alternatives, from Apple Music to YouTube, but in the digital era, Spotify has used data to cement its central cultural position in the music world through music curation and discovery.

Music Curation

Key to Spotify’s success is its focus on playlists. Originally, the feature was standalone: users would create their own personal playlists, but a small music discovery app called Tunigo caught the company’s eye with its curated activity-based playlists (e.g. music for cooking or for a rainy day).

Spotify quickly absorbed the app and began integrating Tunigo into its product. Of course, these curated playlists are a crucial aspect of Spotify as a product now, and that curation’s since expanded to include live events and other revenue streams.

curated playlist example
curated playlist example
curated playlist example

Some of Spotify’s more popular curated playlists.

The most famous of these curated playlists is RapCaviar.

It’s tastemaking at scale, and for those that’ve never heard of it, just know that being featured on RapCaviar essentially guarantees most songs millions of streams and a spot on Billboard’s weekly Hot 100 ranking: as an example, Lil Uzi Vert’s 2017 song “XO Tour Llif3” charted at #10 after appearing on RapCaviar, despite getting little to no radio play—traditionally the most powerful factor in determining a song’s Hot 100 placement, along with sales.

And, RapCaviar’s put together by a single person: Tuma Basa, Spotify’s former global programming head of hip-hop. Basa’s role was to regularly hand-pick songs tracks for RapCaviar and around 30 other playlists, and he relied entirely on listener data to put together these playlists. Metrics like the frequency of artist or track searches or the number of times a song would be skipped or saved to a user’s own playlists drove the decision-making behind RapCaviar. The rise of one of the most controversial rappers in recent memory, XXXTENTACION, could be attributed entirely to his placement on RapCaviar, when Tuma Basa realized XXX was on the verge of mainstream popularity in late 2016.

XXXTENTACION's rise The Google Trend for XXXTENTACION’s popularity.

In an interview with Tuma, writer Craig Marks observes as “[Tuma] stops [their] conversation mid-sentence and grabs his Mac to add a track by rapper Ugly God, ‘Stop Smoking Black & Milds.’ Over the weekend, he says, site search spiked for Ugly God, a leading indicator for Basa of vitality. To make room, he studies the data on a pair of J. Cole songs. One, ‘Change,’ has a fairly high skip rate. With a couple of keystrokes, the song is ethered from the playlist.”

Music Discovery

But, Spotify takes it one step further, beyond curation: personalized playlists. Music discovery is just as important as music curation and at the heart of that is the algorithm for Discovery Weekly, Spotify’s algorithm-backed 30-song playlists that’re unique to every subscribed user. There’re three pieces to the algorithm that generates these playlists: collaborative filtering, natural language processing, and raw audio categorization.

the Discovery Weekly algorithm A breakdown of the Discover Weekly algorithm, adapted from this Quartz article.

There’s a lot of heavy mathematical and computational theory involved, but I’ll give a simple rundown of it at a high level. You know how Netflix recommends you movies and shows based on what other people liked? That’s exactly what collaborative filtering is, except with Spotify’s internal data, like if a user saved a song to their own personal playlist or visited the artist’s page after listening to a song. Natural language processing is a subfield of artificial intelligence that deals with how computers can understand human speech, and Spotify uses that to get a feel for the public perception of an artist, based on what’s been written online, so it can recommend stuff “similar” to what it knows a user likes.

NLP An example of how natural language processing would analyze a friend’s message to me.

And finally, Spotify analyzes the raw audio of a song. By using the same technology behind facial recognition software, songs can be categorized by characteristics like time signature, key, mode, tempo, and loudness. These three pieces, known as recommender models, all connect and fit together to form the Discover Weekly algorithm.

I’ve personally been using Discover Weekly for years, and it’s recommended me several tracks that now rank among my all-time favorites and that I likely would have never heard without Discover Weekly. I’m also an active member of KSDT (UC San Diego’s college radio station, mentioned earlier), both as a DJ and part of the engineering team. This quarter, I’ve been outsourcing playlist creation using Discover Weekly for my radio show: instead of creating a new playlist every week for my show, I’ve been playing a lightly curated version of my Discover Weekly playlist, condensed to fit within KSDT’s constraints as an alternative radio station (not playing any mainstream or highly popular artists) and within the hour slot I have, and that process has been working incredibly well in terms of quality of song selection, as a more personal testimony to the effectiveness of the Discover Weekly algorithm.

Conclusion

Curated playlists like RapCaviar and algorithmically-personalized ones like Discover Weekly explain both Spotify’s dominance in today’s music landscape and how data has driven and continues to drive large-scale musical consumption in the streaming era. As I said earlier, the history of music listening is the history of culture, and now, in the age of algorithms, Spotify has quantified cultural habits and used that data to become a cultural force.

References

Ciocca, Sophia. “How Does Spotify Know You So Well?” Medium, Medium, 21 June 2018.

Marks, Craig. “How a Hit Rap Song Happens Now.” Vulture, 13 Sept. 2017.

Pasick, Adam. “The Magic That Makes Spotify’s Discover Weekly Playlists so Damn Good.” Quartz, Quartz, 22 Dec. 2015.

Safian, Robert. “Spotify’s $30 Billion Playlist for Global Domination.” Fast Company, Fast Company, 4 Sept. 2018.