Budapest Data 2015

The quickest way to learn and evolve infrastructure is by encountering obstacles and being forced to overcome limitations that keep you inches away from project goals. At Spotify, we’ve encountered many of these obstacles and frustrations as we grew our Big Data systems from a few machines in an office closet aggregating played song events for financial reports, to our current 1300 node Hadoop cluster and a complex architecture that plays a large role in many features that you see in our application today and processes PBs of data every week.

A member of Spotify’s Hadoop ‘squad’ will weave in war stories, failures, frustrations and lessons learned to describe the Hadoop/Big Data architecture at Spotify and talk about how that architecture has evolved over time.