Lead Data Engineer

FutureLearn operates a Remote First working practice. A hybrid approach where home is likely to be where you'll spend most of your working time, but you will need to attend our London offices for relevant meetings, engage with and support colleagues.

For over 15 million learners, FutureLearn is the gateway to high-quality lifelong learning – something we’re truly passionate about in our mission to transform access to education. Partnering with over 260 international educational institutions and businesses, we up-skill learners and fire up their creativity, interest – and, of course, their CVs! – with our online courses. We’re building a connected, collaborative, global community where learning is available for all. We’re on the cusp of emerging from a plucky EdTech start-up into a truly transformational business, with meaty goals for the next five years. You could help us achieve those goals, working in a dynamic environment and with a truly passionate team.

What is the opportunity?

The Lead Data Engineer provides leadership to the Data Engineering team. They are responsible for establishing the vision and direction the ETL data pipeline and other data services and infrastructure that supports FutureLearn’s business and products and delivering it.

They manage the team’s priorities to ensure that the correct features are being built in the right order and that work supporting the needs of the strategic teams and Data Scientists is balanced with longer term development of our data infrastructure.

What does success look like?

This Lead Data Engineer develops the vision and strategy of our data architecture, communicates it and leads the team to deliver it. To do this, the Lead Data Engineer, collaborates with other product teams to understand and agree priorities and work together on implementing parts of the ETL and other data engineering projects that would enable them to collect and analyse the data they need and build data driven features.

They also collaborate with the Technology team and Site Reliability Engineer to ensure that the work that they are doing is consistent with the overall technical direction of FutureLearn and that the technical dependencies between the application platform and the data platform can be implemented in the most amenable way for both systems.

The Lead Data Engineer is responsible for building and maintaining FutureLearn’s cloud based data pipeline. This involves:

• Architecting, designing and building a bespoke Extract, Transform, Load (ETL) process built predominantly in Ruby
• Architecting, designing and building ancillary services for the ETL, often web based, and using JSON APIs and socket IPC approaches, in both Ruby and Python.
• Managing our Postgres based Data Warehouse including collaborating with Data Scientists to design and enhance the schema using the Kimball model of star schemas.
• Integrating and maintaining big data solutions and recognising where these are appropriate tools to be used - currently we use Amazon Redshift, Amazon ElastiCache and others.
• Working with Data Scientists to build new machine learning and advanced data mining solutions and integrate them reliably into our production architecture.
• Familiarity with the implementation of hybrid content and collaborative filtering based recommendations engines in order to enhance and support our internal Course Recommendations Engine
• The Lead Data Engineer also line manages, mentors and support the other Data Engineers in the team.

What will you bring to the role?

Excellent knowledge of data platforms and technologies and possess direct experience with some of the data technologies we use. Direct previous experience of working with the ingest, processing and storage of data, ideally in a cloud-based context. You can show understanding of the problems data technologies must solve for an analyst, data scientist or data engineer.

Experience in leading, managing and/or mentoring software engineers as well as leading engineering and/or cross-functional teams is also required

• You will line manage and be responsible for the development of a small team of dedicated data engineers
• You will have demonstrated experience of building software for the cloud, and believe in a DevOps culture
• Strong written and verbal communication skills with proven experience dealing with different levels of an organisation from Executive level down.
• Candidates with direct experience in building data solutions, ETL pipelines, data validation, recommendation engines and machine-learning approaches will be particularly suitable.
• Experience with building Web apps to productionise tools and/or models built by a Data Science team (e.g., Django) would be a bonus.

Technical skills include:

• Amazon Web Services (including EC2, VPC, RDS, IAM, CloudWatch)
• Apache Spark, Amazon Redshift, Airflow and other data technologies
• Confluent Platform (Kafka, Kafka Streams)
• Ruby and Python based API microservices
• Queueing Systems and IPC approaches using ZeroMQ & Sidekiq
• Building ETL pipelines in Ruby/Kiba & PostgreSQL
• Git, GitHub
• Test-driven development (TDD)

What we will offer you?

• 28 days of Annual Leave plus UK Public Holidays
• Roll over up to 5 days Holiday
• Buy and Sell up to 5 days Holiday
• Westfield Health Cash Plan
• Personal ‘recharge’ days, no questions asked
• Cycle to Work scheme
• Season Ticket Loan
• Charity work – 1 day dedicated to a charity of your choice
• Calm Premium Subscription

What happens next?

Ready to go for it? Click 'Apply', include your CV and a cover letter telling us why you'd like to become a FutureLearner, and how you’ll nail this role.

Recruitment Process

Please contact jobs@futurelearn.com if you require any reasonable adjustments or alterations to be made to support you through the recruitment process.

Diversity Statement

We value all the great benefits that diversity brings and encourage everyone to bring their whole self to work – At FutureLearn we are proud to have Diversity and Inclusion at the centre of everything we do. We are committed to Equal Employment Opportunity regardless of race, colour, national origin, ethnicity, gender, age, disability, sexual orientation, gender identity or religion.

We can't wait to meet you! #FutureLearnCareers