Data Engineer
San Francisco, CA or New York, NY
Patreon
Patreon is the best place to build community with your biggest fans, share exclusive work, and turn your passion into a lasting creative business.Do you believe that creators should have the ability to get paid for the value they give to their fans?
We do, which is why we're building Patreon, a platform that powers membership services for creators with established followings. Patreon strives to provide creators with insight, education, and tools that make it possible to retain creative control while running their creative business, so creators can focus on creating and energizing their fanbases.
We have payed out over $500 million directly to creators on our platform this year alone, and our user base has doubled. In order to support this level of growth, we are looking for a Data Engineer.
What you will do:
- Maintain and improve our data warehouse (Redshift) and data lake.
- Design and build batch and streaming pipelines using Airflow, Spark, Kinesis/Kafka, S3, Redshift, ElasticSearch, etc.
- Assist our Data Science and Analytics teams with DAGs, data modeling and data validation.
- Assist our product teams with designing and building out our Analytics Platform, ML Platform and Discovery Platform.
Skills and experience you possess:
- 2+ years experience in a Data Engineering role
- 4+ years experience developing in Python and/or Scala.
- 4+ years working with databases and writing SQL.
- Experience with databases such as Redshift, Snowflake, BigQuery, ElasticSearch. Clickhouse etc.
- Experience building data pipelines using such tools as Airflow, Databricks/EMR, Kafka/Kinesis, Hadoop/Hive, S3, etc.
- Knowledge of infrastructure as code and configuration management systems, such as terraform, ansible, etc.
Projects you may work on:
- Revamp our data warehouse with new access controls and monitoring for improved performance.
- Work with data science to build new analytics pipelines and improve existing pipelines.
- Work with product engineers to build a product-facing analytics platform. This will involve such projects as real time traffic and click aggregation and maintenance of an ElasticSearch cluster.
- Work with ML Engineers to build model training and deployment pipelines.
- Work with search and discovery engineers building out a discovery platform.
What you will have the chance to learn:
- Data Warehouse and Pipelining best practices
- On a team of 5-10 determining the long-term data architecture of Patreon.
- AWS Infrastructure, Machine Learning, Databases, Databricks.
Who you'll work with:
At Patreon, you'll join a high-performing and highly-empathetic team of people who proudly work on fulfilling our mission of funding the creative class. Our culture of creator-first, thoughtful teammates keeps work creative, stretching, and rewarding.
Our Core Behaviors:
- Put Creators First. Patreon is nothing without our creators.
- Achieve Ambitious Outcomes. Set, measure, and accomplish goals that deliver massive value to our creators and patrons.
- Cultivate Inclusion. We want an environment that retains and engages the diverse teams we build.
- Bias Towards Action. When in doubt, we take the next best step, then course correct when needed. We go out of our way to fix problems when we see them. We take ownership seriously.
- Be Candid and Kind. Be extremely caring and extremely direct in all you do at Patreon, especially when it comes to giving positive and constructive feedback.
- Be Curious. You don’t know it all, and that’s the fun part. Everything gets better when you’re curious. Things get more interesting, more clear, and more approachable. When you bring curiosity into the workplace, you’re growing yourself, your teammates, and Patreon as a whole.
Want to learn more about Patreon?
Tags: Airflow Ansible AWS BigQuery Databricks Data pipelines Elasticsearch Engineering Hadoop Kafka Kinesis Machine Learning Model training Pipelines Python Redshift Scala Snowflake Spark SQL Streaming Terraform
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open AI Engineer jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Product Data Analyst jobs
- Open Senior Data Architect jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Principal Data Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open NLP-related jobs
- Open Data governance-related jobs
- Open LLMs-related jobs
- Open Airflow-related jobs
- Open Data warehouse-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs