Senior Data Engineer (Data migration)

We're looking for an experienced Data Engineer for a Swedish multinational telecommunications company and mobile network.

As a Senior Data Engineer, you will play a pivotal role in migrating our data from Hadoop to a new on-premises open-source tools-based data platform.

This role requires deep data engineering expertise, focusing on Spark, Scala, Airflow, Hadoop, and Python.

Responsibilities

• Lead the migration of data from Hadoop to a new on-premises open-source tools-based data platform.
• Design and implement robust data migration processes ensuring data integrity, consistency, and performance.
• Work with Apache Spark, Scala, Airflow, Hadoop, and Python to manage and optimize data workflows.
• Collaborate with cross-functional teams to understand data requirements and migration goals.
• Develop and maintain detailed migration documentation, including data mappings, transformation rules, and processes.
• Troubleshoot and resolve any issues that arise during the migration process.
• Implement best practices for data management, storage, and retrieval.
• Ensure the security and compliance of data throughout the migration process.
• Use GitHub for version control and collaboration on code repositories.

Qualifications

• Extensive experience with Apache Spark, Scala, Airflow, and Hadoop.
• Proficiency in Python for data engineering tasks.
• Strong experience with version control systems, particularly GitHub.
• Proven track record of leading successful data migration projects.
• Excellent problem-solving skills and the ability to troubleshoot complex issues.
• Strong communication and collaboration skills.
• Ability to work independently and lead a team in a fast-paced environment.

What we offer

• 20 days + public holidays.
• Ability to work remotely or join our tech hub in Warsaw.
• Help and support from our caring HR team.
• The exchange of experience and work with talented colleagues.