Staff Data Engineer - Messaging Platform

The Freemium R&D team oversees the entire user journey on Spotify and ensures we engage with people in innovative ways, every step of the way. Our team grows Spotify’s audience by finding future listeners around the world and delivering the right value to them, at the right time. With research, product development, product design, engineering, and marketing all collaborating in one organization, we’re able to quickly create meaningful features and services for millions of people around the world, resulting in joyful, long-lasting relationships with Spotify.
We are looking for a staff engineer to lead strategic efforts across Spotify’s messaging platform. If you have received any push message on the Spotify app it is likely powered by this platform. Our teams build and maintain the platform that delivers billions of messages across email, push, in-app and SMS, and a few more channels. These messages host an important emotional connection with Spotify users, helping them stay up to date with what’s new from their favorite creator, find their next playlist or podcast, or keep informed with new features and value in the Spotify experience.

What You'll Do:

• Build large-scale batch and real-time data pipelines with frameworks like Scio, Google cloud dataflow, BigQuery etc.
• Architect the platform to simplify and evolve different components in collaboration with other staff and senior engineers in the product area. 
• Foster engineering best practices on your team(s) by setting standards and mentoring team members.
• Collaborate with data scientists, ML engineers, Engineering managers and Product managers to identify and solve the problems in the domain.
• You can expect to tackle unique data engineering challenges that include, 
• Delivering high quality datasets to power our machine learning algorithms.
• Building and surfacing performance metrics to operators of the platform.
• Building observability into the system.
• Real time data processing to power time critical messages.
• Effectively maintaining a large fleet of data pipelines and backend services.
• Making the platform self served for developers across the company.

Who You Are:

• You are an experienced technical leader who can work across multiple teams.
• You are comfortable with the JVM and object-oriented programming–experience with Scala, Java, and Python is ideal.
• You have experience with one or more higher-level JVM-based data processing frameworks such as Beam, Dataflow, Crunch, Scalding, Storm, Spark. Scio & Luigi would be an encouraged bonus.
• You have experience in building distributed and high volume services and data pipelines in Java and Scala.
• You are comfortable exploring amorphous projects and can scope the size and feasibility of projects of all sizes.

Where You'll Be:

• We are a distributed workforce enabling our band members to find a work mode that is best for them!
• Where in the world? For this role, it can be within the Americas region in which we have a work location.
• We ask that our team members be located within Central Standard time zone, Eastern Standard time zone, or Brasília time zone for the purposes of our collaboration hours. 
• Prefer an office to work from home instead? Not a problem! We have plenty of options for your working preferences. Find more information about our Work From Anywhere options here.
• Working hours? We operate within the Eastern Standard time zone for collaboration.

Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.
Spotify transformed music listening forever when we launched in 2008. Our mission is to unlock the potential of human creativity by giving a million creative artists the opportunity to live off their art and billions of fans the chance to enjoy and be passionate about these creators. Everything we do is driven by our love for music and podcasting. Today, we are the world’s most popular audio streaming subscription service.