Senior Data Engineer, Data Insights
Remote, United States
Wayfair Inc.
Shop Wayfair for A Zillion Things Home across all styles and budgets. 5,000 brands of furniture, lighting, cookware, and more. Free Shipping on most items.Who we are
As a Senior Software Engineer on the Data Insights team, you will be responsible for developing, owning, and scaling a monitoring and insights pipeline ingesting data from every data platform owned by Wayfair. Your contributions will ensure the continued growth, health, and availability of a dataset that powers every infrastructure decision at Wayfair. This pipeline integrates streaming technology, batch processing, and ML to provide forecasting, anomaly detection, cost insights, and value reporting.
- 70 kafka clusters and over 30k kafka topics across the world
- Thousands of cloud assets, including database tables, storage buckets
- Thousands of dashboards, applications, and user tools
- Millions of user and service interactions with data systems
Candidates that are a fit for this profile are familiar with data engineering or business intelligence, are experienced with Python, systems and configuration as code, and are passionate about building highly resilient, scalable solutions, and would be interested in the following specifics for this team:
What You’ll Do
- Work with partner teams to ingest data from their platforms into a centralized monitoring platform
- Design and implement self-service offerings for frictionless onboarding based on stakeholder requirements
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
- Design, build, and expose value added services on top of this pipeline, such as cost attribution, anomaly detection, automated cost-reduction tools, and more
- Architect and implement a monitoring and alerting approach to enable exposure of Service Level Indicators (SLIs) and exposure/adherence to Service Level Objectives (SLOs)
What You’ll Need
- 5+ years of experience in Data Engineering role
- Applied experience with Python and at least one OLAP database (Redshift, bigquery, snowflake or equivalent) at scale
- Familiarity with Puppet, Chef, Terraform (or similar technologies)
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
About Wayfair Inc.
Wayfair is one of the world’s largest online destinations for the home. Whether you work in our global headquarters in Boston or Berlin, or in our warehouses or offices throughout the world, we’re reinventing the way people shop for their homes. Through our commitment to industry-leading technology and creative problem-solving, we are confident that Wayfair will be home to the most rewarding work of your career. If you’re looking for rapid growth, constant learning, and dynamic challenges, then you’ll find that amazing career opportunities are knocking.
No matter who you are, Wayfair is a place you can call home. We’re a community of innovators, risk-takers, and trailblazers who celebrate our differences, and know that our unique perspectives make us stronger, smarter, and well-positioned for success. We value and rely on the collective voices of our employees, customers, community, and suppliers to help guide us as we build a better Wayfair – and world – for all. Every voice, every perspective matters. That’s why we’re proud to be an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, or genetic information.
Tags: Big Data BigQuery Business Intelligence Data pipelines Engineering Kafka Machine Learning OLAP Pipelines Python Redshift Snowflake Streaming Terraform
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Principal Data Engineer jobs
- Open Sr Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Data Scientist II jobs
- Open Product Data Analyst jobs
- Open Senior Data Architect jobs
- Open Sr. Data Scientist jobs
- Open Business Data Analyst jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open GCP-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open NLP-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Hadoop-related jobs
- Open LLMs-related jobs
- Open Generative AI-related jobs
- Open Databricks-related jobs