Data Engineer

This role is 6 months Contract

What you’ll do

• To support implementation of projects focused on collecting, aggregating, storing, reconciling, and making data accessible from disparate sources to enable analysis and decision making.
• Ensuring stakeholders can access and manipulate data for routine and ad hoc analysis.
• To support the full lifecycle of data from ingesting through analytics to action.
• Support planning and delivery of data warehouse and storage architecture.
• Support the planning and implementation of data design services, providing sizing and configuration assistance, and performing needs assessments.
• Develop and maintain data warehouse schematics, layouts, architectures, and relational databases for data storage and data mining
• Customize data storage and extraction, data mining, database architecture, metadata and repository creation.
• Implement effective metrics and monitoring processes.
• Participate in the design and development of business intelligence reporting tools.
• Program and maintain report forms and formats, information dashboards, data generators, canned reports and other end-user information portals or resources.
• Knowledge of database, storage, collection and aggregation models, techniques and technologies, with the ability to apply such methods to solve business problems.
• Knowledge of structured problem-solving assignments and strong project management and people management skills
• Knowledge of SharePoint, PowerBI, PowerPivot, SRSS, Excel skills (with embedded Pivot Tables & Macros) with SQL Skills

Specific Outcomes

• Assist with prioritising and validating business demand for data and reporting
• Lead and support working sessions to analyse new demand and estimate the effort to integrate for the reporting data structure
• Participate in the design and development of business intelligence reporting.
• Build SQL scripts processes from ingestion to consumption of analytics
• Response and resolution for implemented changes
• Proven experience in handling Unstructured data
• Deployment and change support
• Customize data storage and extraction, data mining, database architecture, metadata and repository creation.
• Support the implementation effective metrics and monitoring processes.
• Build and maintain report forms and formats, data generators, canned reports and other end-user information portals or resources.
• Review and monitor integration and ETL jobs

Requirements

• Min 3 to 5 years experience with experience in deploying real time pipelines
• Possess a Bachelor's degree in IT and Applied Mathematics, Statistics, or another relevant field
• Experience with pipeline and workflow management tools.
• Strong hands on expertise with any of programming language is a must
• Advance working SQL knowledge, experience and execution
• Familiar with containerisation and orchestration tools
• Familiar working with ETL using either Spark, PLSQL, Docket etc
• Experience in building using open source ETL/ELT tools such as Airflow, Nifi, Airbyte, Oozie
• Data Lake high level principles and understanding of Big Data ecosystems
• Data warehousing principles and business intelligence understanding