Data Infrastructure Engineer

• Collaborate with Data Engineer to design and implement scalable and reliable systems for ingestion, processing, and analysis of large disparate data sets from diverse sources.
• Improve existing and create new data infrastructure components to better automate extraction, transformation, loading, and other data management processes.
• Develop tools and applications to proactively measure, monitor, and improve data quality and consistency during loading and analysis processes.
• Analyze and improve efficiency, reliability, and scalability of data infrastructure and processes.
• Work with data team to define and promote best practices for data management and analysis, and to build and improve systems to implement and support these practices.
• Maximizing the reliability and performance of services used in data infrastructure.

Requirements

• 4+ years of relevant work experience in data infrastructure and software engineering.
• Strong scripting and development proficiency in two or more languages (Python preferred, experience in Airflow DAG is a plus).
• Professional experience with multiple relational and non-relational database systems.
• Desire and readiness to take on greater ownership of and responsibility for data infrastructure.
• Professional experience retrieving, parsing, cleaning, and transforming data from multiple formats (e.g. XML, JSON, CSV, PDF, etc.) and delivery mechanisms (e.g. files, streams, APIs).
• Experience with cloud infrastructure services, especially Google Cloud Platform.
• Experience with development pipeline and continuous development with Git and its automation.
• Experience with Kubernetes and has managed the deployment and services inside Kubernetes.
• Familiarity with basic cloud networking.