The BBC Audience Platform Data team collects, transforms and delivers billions of events each day from audience interactions with mobile apps and web sites such as BBC News, BBC Sport and iPlayer. This presentation shares some big data experience of the 7 critical steps the BBC took to migrate from Spark clusters with Map-Reduce to a serverless architecture. It covers the good, the bad and the ugly, including lessons learnt along the way. These steps are also applicable to new architectures. This big data architecture has been running in production for nearly a year and have achieved the goal of real-time analysis at up to 25000 transactions per second. This allows the BBC to meet the demands of even the biggest news days, as well as seeing significant savings on its cloud infrastructure.
Video producer: http://www.devoxx.com/