Sean Braithwaite is a data scientist based in Berlin. For the past 8 years he’s been using data for everything from data driven art installations to real time ad bidding. Most recently he’s been responsible for scaling SoundClouds data pipeline to handle billions of events per day.
Data Pipelines as Data Structures
Collecting, transferring and transforming immutable event data has become the foundation of every data driven organisation. The design of these data pipelines is often hacked together to cross environments, programming languages and teams. This talk will present the key principals of designing data pipelines to survive complexity of a growing data hungry organisation.