Senior Data Engineer

CAPCO POLAND

Capco Poland is a global technology and management consultancy specializing in driving digital transformation across the financial services industry. We are passionate about helping our clients succeed in an ever-changing industry.

We also are:

• Experts in banking and payments, capital markets, wealth and asset management
• Focused on maintaining our nimble, agile, and entrepreneurial culture
• Committed to growing our business and hiring the best talent to help us get there

Project:

Our Capco team is partnering with one of the our Client -  Global Finance Organization to deliver an ecosystem of curated, enriched and protected sets of data – created from global, raw, structured and unstructured sources. We are utilising the latest technologies to solve business problems and deliver value and truly unique insights

THINGS YOU WILL DO: 

• Collecting, storing, processing, and analyzing of large sets of data
• Choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. 
• Responsible for integrating them with the architecture used across the company and to help build out some core services that power Machine Learning and analytics systems. 

 We are looking for Data Engineers who have:

• Ability to process and rationalise structured data, message data and semi/unstructured data and ability to integrate multiple large data sources and databases into one system 
• Proficient understanding of distributed computing principles and of the fundamental design principles behind a scalable application 
• Strong knowledge of the Big Data eco system, experience with Hortonworks/Cloudera platforms  
• Practical experience in using HDFS 
• Practical expertise in developing applications and using querying tools on top of Hive, Spark (PySpark) 
• Strong Scala skills 
• Experience in Python, particularly the Anaconda environment and Python based ML model deployment 
• Experience of Continuous Integration/Continuous Deployment (Jenkins/Hudson/Ansible) 
• Experience with using GIT/GITLAB as a version control system. 
• Experience in working in Teams using the Agile Methods (SCRUM) and Confluence/JIRA  
• Good communication skills (written and spoken), ability to engage with different stakeholders and to synthesise different opinions and priorities.  

 Nice to Haves 

• Knowledge of at least one Python web framework (preferably: Flask, Tornado, and/or twisted) 
• Basic understanding of front-end technologies, such as JavaScript, HTML5, and CSS3 would be a plus 
• Good understanding of global markets, markets macrostructure and macro economics  
• Knowledge of Elastic Search Stack (ELK) 
• Experience with Google Cloud Platform (Data Proc / Dataflow) 

Other

• Preferable knowledge and experience in Data Quality & Governance. 
• Experience of big data programmes preferable 
• Enthusiastic and energetic problem solver to join an ambitious team 
• Attention to detail 
• Good knowledge of SDLC and formal Agile processes, a bias towards TDD and a willingness to test products as part of the delivery cycle 
• Ability to communicate effectively in a multi-programme environment across a range of stakeholders