Data Engineer
Birmingham, GB, B37 7ES
LRQA
We help businesses evolve by connecting them with tomorrow’s thinking, today.Role Purpose
The Data Engineer position is an integral part of the Data team within the Digital and Information function. This role entails overseeing the availability, integrity, quality, and accuracy of data. The primary responsibilities include designing, developing, and maintaining data pipelines and workflows to support the organisation's data processing needs and facilitate BI reporting and analytics delivery. The Data Engineer will contribute to enhancing our Azure-based data platform using technologies such as Azure Data Factory and Azure Databricks and will have the opportunity to work on exciting digital transformation projects.
Operating within Agile principles and methodologies, the Data Engineer collaborates with other data engineers and key stakeholders to shape and manage the portfolio of reporting and data requirements. Working alongside peers, this role contributes to refining the overall data strategy.
Key Responsibilities
-
Design, develop, and maintain scalable data pipelines and workflows using Azure Data Factory and Databricks.
-
Perform data transformation and integration tasks to consolidate and enrich data from various sources into a unified format, whilst also ensuring data governance and compliance by design
-
Implement data quality checks and validation processes to ensure accuracy, completeness, and consistency of data.
-
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements, perform UAT and deliver solutions accordingly.
-
Monitor pipeline performance and troubleshoot issues promptly to ensure uninterrupted data processing.
-
Document data pipelines, workflows, and processes to ensure knowledge sharing and maintainability.
-
Ensures the effective & timely management and closure of 2nd line data and reporting support requests, through the appropriate ticketing systems, to minimise impact on the operations of the wider business.
-
Actively participate in daily stand ups, weekly stakeholder meetings, retrospectives.
Requirements
-
Hands-on Experience with Azure Data Factory: Proven experience in designing, developing, and managing data pipelines using Azure Data Factory.
-
Good knowledge of Databricks for data processing and analytics
-
Experience with data modelling techniques and data warehousing concepts.
-
Proficiency in SQL for data querying and manipulation.
-
Ability to work effectively in a fast-paced, dynamic environment and manage multiple priorities, following the Agile way of working
-
Problem-solving Skills: Ability to analyse complex data problems and propose effective solutions.
-
Communication and Collaboration: Good communication skills with the ability to collaborate effectively with cross-functional teams.
-
Experience with Python is highly desirable
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Azure Databricks Data governance Data pipelines Data quality Data strategy Data Warehousing Pipelines Python SQL
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Marketing Data Analyst jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Principal Data Engineer jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Sr Data Engineer jobs
- Open Business Data Analyst jobs
- Open Data Analytics Engineer jobs
- Open Data Analyst Intern jobs
- Open Product Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Data Quality Analyst jobs
- Open Manager, Data Engineering jobs
- Open Research Scientist jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Generative AI-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs