Senior Data Engineer III

Wayfair Data Engineering is the engine that powers our data obsessed eCommerce enterprise. We move fast, iterating quickly on big business problems. We work smart, applying technology to unlock insights and provide outsized value to our customers. We swing big, knowing our customers won't benefit from micro optimizations. Leveraging the largest data set for products sold in the Home space, this team treats data as an asset and determines how to maximize its business value and extend our competitive advantage. 

As a Senior Data Engineer, you will work collaboratively with our Catalog Engineering teams to develop the data infrastructure which enables our leadership team to make strategic decisions at the speed of thought through self service reporting.You will be instrumental in building out the data architecture to ensure we capture the full who, what, where, when, and why as our data is generated, transformed, and mutated over time, working to build a cohesive, scalable, accurate, and performant foundational source of truth upon which all data & analytics users across the company will build. 

What you’ll do

• Build data pipelines to assemble large, complex sets of data that meet non-functional and functional business requirements
• Build analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
• Work closely with data architect, SMEs and other technology partners to develop & execute on the data architecture roadmap for the pricing functional area 
• Work with stakeholders including the Executive, Product, Customer teams to support their data infrastructure needs while assisting with data-related technical issues
• Be a technical leader to junior engineers, ensure the team is being challenged, expose your team to new opportunities, while still being able to deliver on ambitious goals.
• Act as a subject matter expert to leadership for technical guidance, solution design and best practices.
• Design and implement big data architecture, data pipelines, reporting, dashboarding, data exploration, processes and quality controls to enable self-service Business Intelligence.
• Keep current on big data and data visualization technology trends, evaluate, work on proof-of-concept and make recommendations on the technologies based on their merit.

Who You Are

• A true expert on big data, comfortable working with datasets of varying latencies and size and disparate platforms
• Excited about unlocking the valuable data hidden in inaccessible raw tables and logs 
• Attentive to detail and with a relentless focus on accuracy
• Excited to collaborate with partners in business reporting and engineering to determine the source of truth of key business metrics
• Excellent communication and presentation skills, strong business acumen, critical thinking, and ability to work cross functionally through collaboration with engineering and business partners. 
• Demonstrated success working with stakeholders and implementing end-to-end data solutions including data model, data pipelines and Business Intelligence products to solve complex business requirements.

What You Have

• 5+ years of experience in designing and implementing large-scale global Data warehouse architectures, OLAP technologies, Star-schema, Snowflake schema and Aggregation Techniques.
• 3+ years of hands-on experience of Data Lake/Hadoop platform and performance tuning Hadoop/Spark implementations.
• Experience with one or more relevant tools (Sqoop, Flume, Kafka, Avro, Parquet) and SQL-on-Hadoop technology (Hive, Spark SQL, Presto)
• Experienced developing in cloud platforms such as Google Cloud Platform (preferred), AWS, Azure, or Snowflake at scale.
• Experience with real-time data streaming tools like Kafka, Kinesis, Apache Storm or any similar tools.
• Experience in development of custom built BI and big data reporting solutions using tools like DataStudio, Looker, Tableau, AtScale, PowerBI, Qlik, or any similar tools.
• Experience architecting database management systems (such as  Oracle, MySQL, PostgreSQL, MS SQL Server) and analytical technologies in the industry including MPP (Vertica, Teradata, Redshift)  and NoSQL databases (such as Cassandra, MongoDB, Redis, Aerospike). Prior experience with databases 100TB+ highly desired.
• 3+ years of  programming experience with at least one language such as Python, Scala, Java, C++  or other modern OOP programing language
• Advanced SQL proficiency highly desired.
• Bachelors or Masters in Computer Science, Computer Engineering, Analytics, Mathematics, Statistics, Information Systems, Economics, Management or other quantitative discipline fields with strong academic record.

What we’d love to see (but isn’t required)

• Ecommerce or retail analytics experience a strong plus
• Experience with code management tools (e.g. Git, SVN) and DevOps tools (e.g. Docker, Bamboo, Jenkins)
• Experience in designing data engineering solutions using open source and proprietary cloud data pipeline tools such as Airflow, Glue and Dataflow
• Certification in one of the cloud platforms (AWS/GCP/Azure)

About Wayfair Inc.

Wayfair is one of the world’s largest online destinations for the home. Whether you work in our global headquarters in Boston or Berlin, or in our warehouses or offices throughout the world, we’re reinventing the way people shop for their homes. Through our commitment to industry-leading technology and creative problem-solving, we are confident that Wayfair will be home to the most rewarding work of your career. If you’re looking for rapid growth, constant learning, and dynamic challenges, then you’ll find that amazing career opportunities are knocking.

No matter who you are, Wayfair is a place you can call home. We’re a community of innovators, risk-takers, and trailblazers who celebrate our differences, and know that our unique perspectives make us stronger, smarter, and well-positioned for success. We value and rely on the collective voices of our employees, customers, community, and suppliers to help guide us as we build a better Wayfair – and world – for all. Every voice, every perspective matters. That’s why we’re proud to be an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, or genetic information.