Senior Data Engineer

Job Description & Summary

We are seeking a highly skilled Senior Data Engineer with extensive experience in data engineering. The ideal candidate will have a strong background in SQL, proficiency in data visualization tools such as Power BI, and hands-on experience with PySpark and Scala Spark. Prior experience with the Quantexa platform and working with AWS is essential. Additionally, a solid understanding of DevOps is also a must.

Responsibilities:

• Design, develop, and maintain robust and scalable data processing pipelines using PySpark and Scala Spark;

• Collaborate with cross-functional teams to define, design, and implement new features;

• Cleanse, parse, and standardize global datasets;

• Perform data classification and entity extraction/resolution;

• Optimize and improve data processing performance through big data testing;

• Utilize pre-packaged code for processing and utilizing third-party data sources;

• Ensure data quality and integrity through efficient data manipulation;

• Help implement and configure Continuous Integration (CI) and Continuous Delivery (CD) tooling;

• Help deploy updates and fixes, build and maintain highly available systems;

• Implement and improve monitoring and alerting;

• Perform root cause analysis for production errors and investigate/resolve technical issues;

• Work in an Agile environment and contribute to the continuous improvement of our development processes.

Requirements:

• Master’s degree in a related field;

• At least 6 years of hands-on experience in data engineering;

• Strong proficiency in SQL;

• Proficiency in data visualization tools such as Power BI;

• Solid experience with big data technologies like PySpark and Scala Spark;

• Hands-on experience with the Quantexa platform is a must;

• Prior experience working with AWS;

• Familiarity with Agile development methodologies;

• Strong problem-solving skills and attention to detail;

• Excellent communication and teamwork abilities;

• Experience with Airflow for workflow automation and scheduling;

• Focus on Docker/Kubernetes;

• Strong Linux system administration background;

• DevOps tooling/automation written with Bash, Python, Groovy, Jenkins;

• Experience with provisioning software/frameworks like Elasticsearch, Spark, Hadoop, PostgreSQL;

• Knowledge of containerization technologies like Docker and orchestration tools like Kubernetes;

• Experience working with Spring Boot Java framework.

We offer:

• Interesting career development options and challenging assignments for top Romanian and international clients;

• International mobility and short/medium-term projects abroad;

• A professional and team-oriented work environment;

• Great colleagues to work with;

• Up-to-date technologies and methodology;

• Professional development and training opportunities in Romania and abroad.