Data Engineer

- - - - - - - - - - - -

Job Description:

Requirements:

• Strong problem-solving techniques and analytical skills.
• Proficient in at Python module like Pandas/NumPy/Sci-kit.
• Strong Experience in designing and constructing Notebooks for Data Transformation.
• A strong experience in relational Database knowledge is a must and knowledge on any Database system hosting ERP data will be added advantage.
• Strong experience in writing SQL Queries.
• Technical expertise on building ETL data pipeline, extraction from different source like Azure storage, HDFS, Kafka topics, structured and unstructured files, Hive.
• Good understanding how to apply technologies to solve big data problems and to develop innovative big data solutions. To be able to do this, the Data Engineer should have extensive knowledge in different programming or scripting languages [Python]
• Good understanding on Azure components (Azure Data Factory, Azure DataBricks, Azure Daka Lake Storage)
• Good understanding Version controlling (Git, GitHub, azure DevOps).
• Competency in Cloud environment is must [preferred, Azure]
• Should have experience in transforming raw/unstructured data to clean data [Data Quality]
• Experience in Data Monitoring on Data Lake will be added advantage.

Tools and Technology

Primary :

• Python
• Databricks
• Azure
• CI/CD Deployments

Secondary :

Jira

Lean and Agile

PowerBI

PowerApps