Senior Data Engineer
New York, NY
Come see what’s cookin’ at HelloFresh!
At HelloFresh, we want to revolutionize the way we eat by making it more convenient and exciting to cook meals from scratch. We have offices all over the world and we deliver delicious meals to millions of people.
We are the industry leader in meal-kit subscription services and we’re growing all the time. We have distinct meal-kit services that cater to everyone with the most menu variety in the market, which allows us to reach an incredibly wide population of people.
The HelloFresh team is diverse, high-performing, and international, and our work environment is an inspiring space where you can thrive as a result.
We are hiring a Senior Data Engineer to lead Data Infrastructure & Automation workstreams as a key member of the Data Engineering team within Operations Analytics.
We are growing our Data Engineering capabilities throughout our Supply Chain in order to take our data modeling and automation to the next level by leveraging big data tools and platforms (e.g. Airflow, Spark, Databricks etc.). Within the Operations Analytics team, you will work closely with fellow data engineers, data scientists, and analysts to build and maintain data products that improve the HelloFresh customer offering and enhance efficiency of our Operations.
Our vision is to develop and maintain a best-in-class automated data platform that supports our growing Supply Chain, in delivering the perfect product, at the right time, to the doorstep of all our customers. We are looking for someone who connects stakeholders to data through key projects, while simultaneously focusing on mentorship and developing internal talent.
You will ...
- Data modeling: design, extend and update the HelloFresh’s data model (create new schemas, fact tables, (mat) views, etc.)
- Data infrastructure: Develop, deploy, monitor, and maintain ETLs on Airflow & Databricks
- Data Cleaning: keep data clean and accurate with data sources (Internal & 3rd Party)
- Data Transformations: implement the logic of the data pipeline (aggregations, projections, selections, etc …)
- Design and implement end-to-end data products and Supply Chain automation flows: from data ingestions for data science modeling to creation of automated pipelines to external software (WMS, TMS, SAP etc.)
- Familiar with data architecture, data governance and machine learning model implementations
- Very organized and a structured problem solver with an articulated thought process
- Able to develop an in-depth understanding of HelloFresh’s core product and architecture, and act as an ambassador for state of the art software solutions and industry best practices, offering support and mentorship to colleagues
- Comfortable operating in fast-paced, dynamic environments and working autonomously
- Intellectually curious; don’t mind questioning and challenging the status quo
- A team player who efficiently collaborates with peers and stakeholders
At a minimum, you have...
- MSc in a STEM discipline
- 5+ years’ data engineering experience is required
- Advanced Python (functional and OOP) and SQL skills (DDL, DML, CTEs, query optimization, ...), as well as experience with Apache Spark required
- Knowledge of data structures (DataFrames, RDDs, Dataclasses) and data formats (CSV, JSON, Parquet, Avro, ORC)
- Required experience installing data architectures on Cloud providers (e.g. AWS), using DevOps tools and automating data pipelines.
- The ability to design, implement and deliver maintainable and high-quality code using best practices (e.g. git/github, secrets, configurations, yaml/json)
- Experience with data modeling, design patterns, building highly scalable and secured solutions preferred
- 2+ years experience with job orchestration tools like Airflow or similar
- Experience with end-to-end testing of data pipelines and implementing Data Quality checks (e.g. using frameworks AWS Deequ) is preferred.
- Experience with Dash (and/or Shiny) is a plus
- Competitive Salary & 401k company match that vests immediately upon participation
- Generous parental leave of 16 weeks & PTO policy
- $0 monthly premium and other flexible health plans effective first day of employment
- 75% discount on your subscription to HelloFresh (as well as other product initiatives)
- Snacks, cold brew on tap & monthly catered lunches
- Company sponsored outings & Employee Resource Groups
- Collaborative, dynamic work environment within a fast-paced, mission-driven company