Senior Data Engineer

We are looking for a motivated & results driven data engineer to join our Reporting & Analytics Development Team; delivering reporting and analytical solutions for both new and existing projects including, but not limited to: Lake House architecture and solution development, performance optimization, data feeds development, and opportunities to contribute to Machine Learning & A.I initiatives. You will contribute to bringing the product up to modern cloud and tool stack. You will strive to achieve maximum success within your team, and work together and improve as a unit.

Requirements

• Design and maintain data lake house architecture based on variety of database engines such as MS SQL, Snowflake, Exasol, S3
• Help on planning the growth of the infrastructure; improving system resilience, performance, and stability
• Develop and maintain ETL pipelines from various sources like RDBMS, NoSQL, Kinesis and other batch or streaming sources and flat files
• Utilize AWS serverless architecture patterns in system design (ECS, EMR, Lambda, etc)
• Automate infrastructure with Docker and CI/CD pipelines
• Ensuring consistency of technology usage across the data domain, by continuously reviewing existing toolsets and code and suggesting re-use of components
• Collaborate with data analysts and data scientists to identify requirements and develop the necessary data workflows/models
• Contribute to content recognition AI/ML projects and other initiatives
• Work with stakeholders on requirements and negotiate the best outcome

Required Competencies, Skills & Experience

• Leadership skills
• Designing, building and launching highly available, distributed systems of data extraction, transformation and loading of datasets
• Proficiency with Python/Java, shell scripting, system diagnostic and automation tooling
• Proficiency with writing, optimizing and profiling SQL
• Proficiency with cloud services (AWS)
• Proficiency with Data Warehousing design principles and best practices
• Proficiency with data modelling
• Experience with No-SQL technologies
• Familiarity with CI/CD process as well as tools and processes including Git, Jira
• Experience with setup and configuration, setting up design best practices, coding best practices and configuration management process for the data engineering team as well as with BI tools
• A self-starter with the ability to work effectively in teams
• A curious problem-solving mind set
• Strong attention to detail
• Ability to take ownership of the deliverables
• Ability to communicate with a range of technical and non-technical stakeholders