Data Engineer
Remote
Applications have closed
Wikimedia Foundation
Imagine a world in which every single human being can freely share in the sum of all knowledgeSummary
The Wikimedia Foundation is looking for a Data Engineer to join our team, reporting to the Director of Data Engineering. As a Data Engineer, you will be responsible for building, maintaining and expanding the shared data infrastructure that powers a big part of decision making in the Foundation as well as the Wiki Movement. This includes everything from building scalable pipelines using big data technology to defining and creating a suite of datasets which adhere to our privacy principles.
We are a fully remote, internationally distributed team. We see each other in person 1-2 times a year during one of our off-sites (the last few have been in places like Copenhagen, Majorca and New York) or Wikimania, the annual international conference for the Wiki community.
Open to candidates located in timezones UTC-8 to UTC+2, daily team stand ups occur at 9am UTC-8.
You are responsible for:
- Integrating data from multiple sources to gain insights in areas such as content, traffic, editors, readership and fundraising
- Building scalable data pipelines in collaboration with other data engineers as well as teams across the foundation including product analytics, platform engineering, survey, research and machine learning teams
- Designing the shared data platform that supports use cases for critical aspects of the Wikimedia mission: harassment prevention, image classification, bot detection, DDoS attacks flagging and many more
- Building and maintaining public metrics and datasets
- Implementing data quality monitoring that alerts the team of possible data issues
- Implementing a data governance and lineage solution for all Wikimedia data
Skills and Experience:
- 2+ years of relevant industry experience
- Advanced working knowledge of SQL, relational databases, query authoring, ideally in a variety of flavors (in our team alone we deal with MariaDB, HiveQL, CassandraQL, Spark SQL and Presto)
- Experience with one or more programming languages such as Python, Scala, and Java
- Experience building data pipelines using tools such as Airflow, Spark, Gobblin, Oozie, Yarn
- Familiarity with stream processing systems using Kafka, Spark streaming and/or Flink
- Excellent written and verbal communication skills
- Strong interpersonal and collaboration skills
- BS or MS degree, preferably in Computer Science, or equivalent work experience
Qualities that are important to us:
- Commitment to the mission of the organization and our values
- Commitment to our guiding principles
- Commitment to diversity, equity, and inclusion
- Cross-cultural sensitivity and awareness
- Collaborative working experience
Additionally, we’d love it if you have:
- Experience with Hadoop
- Understanding of related disciplines including Machine Learning, Statistics, Privacy and Algorithms
- Experience working with site reliability engineers
About the Wikimedia Foundation
The Wikimedia Foundation is the nonprofit organization that operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge freely. We host Wikipedia and the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive.
The Wikimedia Foundation is a charitable, not-for-profit organization that relies on donations. We receive donations from millions of individuals around the world, with an average donation of about $15. We also receive donations through institutional grants and gifts. The Wikimedia Foundation is a United States 501(c)(3) tax-exempt organization with offices in San Francisco, California, USA.
As an equal opportunity employer, the Wikimedia Foundation values having a diverse workforce and continuously strives to maintain an inclusive and equitable workplace. We encourage people with a diverse range of backgrounds to apply. We do not discriminate against any person based upon their race, traits historically associated with race, religion, color, national origin, sex, pregnancy or related medical conditions, parental status, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other legally protected characteristics.
If you are a qualified applicant requiring assistance or an accommodation to complete any step of the application process due to a disability, you may contact us at recruiting@wikimedia.org or +1 (415) 839-6885.
More information
U.S. Benefits & Perks
Wikimedia Foundation
Applicant Privacy Policy
News from across the Wikimedia movement
Blog
Wikimedia 2030
Our Commitment to Equity
This is Wikimedia Foundation
Facts Matter
Our Projects
Our Tech Stack
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow Big Data Classification Computer Science Data governance Data pipelines Data quality Engineering Flink Hadoop HiveQL Kafka Machine Learning MariaDB Nonprofit Oozie Pipelines Privacy Python RDBMS Research Scala Spark SQL Statistics Streaming
Perks/benefits: Career development Equity
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open MLOps Engineer jobs
- Open Lead Data Analyst jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Sr Data Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Power BI Developer jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Data Scientist II jobs
- Open Senior Data Architect jobs
- Open Product Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Business Data Analyst jobs
- Open Manager, Data Engineering jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Data Quality Analyst jobs
- Open Data Product Manager jobs
- Open Azure Data Engineer jobs
- Open ETL Developer jobs
- Open Junior Data Engineer jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Java-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs
- Open LLMs-related jobs
- Open DevOps-related jobs