Once we had a nice clean dataset, we wrote a PHP script to fetch 30 second previews of each song with the 7digital API. Not all songs were available in the 7digital catalogue but we got enough to continue with the next step in the project: creating one long seamless remix of all the songs.

To do this we used a Python script by The Echo Nest, called Capsule, which uses the Echo Nest Remix API to automatically arrange and mix a list of songs by beat matching, time stretching and cross-fading. After some initial problems with getting the script to run due to varying bitrates in the preview MP3s, we eventually managed to generate a nice long remix of the Billboard charts starting with “Semper Fidelis“ in 1890 and going all the way up to “Aston Martin Music“ in 2011 (listen to the whole remix on Soundcloud).

Now that we had a remix done we wanted to explore a way of representing the dataset visually. None of us had done anything using D3.js before so this seemed like a good opportunity to give it a try. We built a small single-page webapp using Backbone.js (with Yeoman providing the scaffolding) and added D3 scatterplots to the views. Once we had some simple scatterplots showing, we decided to extend the dataset to include the top five songs from each year to make it a bit more interesting.

Although the original dataset contained a lot of interesting data already, we wanted to augument it further with data from the Echo Nest API. Using the Echo Nest song profile method we fetched metadata and acoustic properties for each song. Again, not all songs in the original dataset were recognised by Echo Nest, but a large proportion were.

With the extra Echo Nest data, the scatterplot visualisations suddenly became more interesting. Now we could use the visualisation to answer questions such as “has the danceability of the top Billboard songs changed over time?” or “were the songs longer or shorter back in the day?”.

The last step was to connect the remix and the visualisation of the dataset, the idea being that you should be able to listen to the remix with the currently playing song highlighted in the plot. This would allow exploration of the dataset in two ways, visually and aurally. The remix is unfortunately limited to the top charted song from each year, but we’ve added links to Spotify, Tomahawk, and 7Digital to all songs where available.

Have a play around and let us know what you think, and if you’re interested the code is on Github.