Data Engineer Lead
Ampol SG, Singapore, Singapore
Ampol
Ampol connects our proud history with everything our business is known for today, our trusted high-quality products, our commitment to customer service, local communities and the largest leading transport fuels network in Australia.About Ampol Trading & Shipping
Our purpose at Ampol is to power better journeys today and tomorrow. Our Trading and Shipping team is key to achieving this, not only supplying our customer’s needs today, but evolving as the energy transition takes place to supply our customers well into the future.
We take pride in our people and value innovation, sustainability, and customer satisfaction. If you’re a professional committed to excellence, Ampol is looking for you.
About the role
This position will be responsible for working with large and disparate data sets, internal and external, quantitative, and qualitative, structured and unstructured to support the management of data across the entire data lifecycle. This role requires a combination of technical expertise, leadership skills and strong focus on data governance practices to ensure the successful delivery of the Data Product Roadmap.
The data engineer brings expert understanding of data and modern data processing technologies to prepare and deliver data assets which support high priority projects.
This position is a part of the Data Product Team, which is responsible to delivering on the prioritised T&S Data roadmap to meet strategic objectives. aligned to the Trading & Shipping organisation.
Key responsibilities
Accountabilities
- Working with the key stakeholders to define data requirements for projects
- Perform data acquisition/ingestion from internal and external sources, including both structured and unstructured data sources (incl. multi-dimension data, text/documents, geospatial data and graph)
- Performing descriptive analytics and data visualisation to share data insights or validate data with stakeholders. Prepare data quality reports
- Develop data transformations and modelling to prepare data sets for consumption for various end uses including reporting or advanced analytics.
- Develop and maintain productionised data outputs/products to support ongoing uses
- Working in a highly agile and flexible environment to deliver outputs in short time frames
- Document technical specifications of data pipelines and technical products delivered
- Develop working understanding of Ampol systems, data, and uses, as a data SME
- Data Strategy and Planning- Continuously develop and enhance Ampol data strategy and architecture and roadmap to ensure alignment to business requirements and overall company vision.
- Data Exploration and Analysis- Explore data to identify patterns, trends and insights. Perform quantitative and qualitative analysis to answer specific business questions and to gain a deeper understanding of data.
- Data Pipeline Development- Build and maintain data pipelines to move and transform data from various sources into Snowflake with assurance that data ingestion is reliable, scalable and optimized for performance.
- Data Modelling and Design- With requirements from business, design and implement data models within Snowflake, including schema design, table structures and data partitioning to support high performance, low latency querying and analytics.
- Data Quality and Integrity- Enforce data quality checks and validation during data ingestion process to ensure accuracy and integrity of data in Snowflake.
- Support Continuous Improvement in Data Product- Support the Commercial Analytics in advancing continuous improvement in the data space, including team engagement to generate awareness of initiatives and track progress.
- Monitoring and Maintenance- Proactively monitor data pipelines and the overall health of Snowflake data infrastructure to identify and resolve issues.
- Automation and Orchestration- Automate data workflows and orchestrate data processes to minimize manual intervention and improve operational efficiency.
- Collaboration with Data Users- Work closely with data analysts, data engineers, and business users to understand their data requirements and provide necessary data support.
Capabilities
- Big Data Engineering – familiarity in designing and developing data processes on cloud based big data platforms
- Data Quality – experience in managing data quality controls to support the completeness and accuracy of data
- Strong understanding of data structures and designing conceptual, logical and physical data models
- Understanding of Information Management, Data Governance, Data Security and Privacy concepts (covered by respective Ampol policies)
Qualifications & experience
- Tertiary qualification in Business, IT, Engineering, Finance or or related discipline or relevant operational experience.
- Project 10+ years of technology and commercial delivery experience with 3 years in a leadership role
- Proficient in digital delivery, technologies and tools
- 3 to 7 year’s downstream oil industry and/or petroleum trading companies in areas such as Supply planning, Supply operations, line finance roles, corporate planning and strategy, and distribution and sales leadership positions.
- Application of agile project methodology
Skills & Experience
- Tertiary qualifications in Computer Science or similar.
- Qualifications or experience in mathematics or statistics highly regarded
- 2+ years of industry experience working as a data engineer to develop data products using SQL, Python, and Spark
- Data modelling experience including ETL
- Data quality management
- Experience with a range of Data platforms (Hadoop, Spark, SQL Databases)
- Experience performing descriptive analytics and developing data visualisation (Power BI, Tableau, Shiny, etc)
- Experience working to support advanced analytics products is desirable
Knowledge & Skills
- Expertise with SQL, Python, and Spark
- Understanding of git flow and relevant code management principles
- Expertise in establishing and monitoring data pipelines and workflows
- Expertise working with varied Relational Databases
- Tertiary qualification in Data Analytics or related discipline
- Project 3 to 7 years of data engineering in Snowflake
- 3 to 7 year’s downstream oil industry and/or petroleum trading companies in areas such as Supply planning, Supply operations, line finance roles, corporate planning and strategy, and distribution and sales leadership positions.
- Proficiency in programming languages like SQL to work with database sets and data manipulation.
- Solid understanding of database management systems, data storage, data indexing and query optimization for efficient data handling.
- Proficient in data modelling techniques and design patterns to help create efficient and scalable data architectures.
- Effective communication skills and has ability to collaborate with cross functional teams to understand data requirements and deliver valuable insights.
- Understand data quality principles and data governance best practices.
Desirable:
- Working knowledge and understanding of advanced analytic techniques (statistical and predictive modelling and machine learning)
- Experience working with Snowflake and/or Talend
- Understanding of graph or noSQL technologies
- Familiarity with Ampol business domains
- Experience with Docker, Kubernetes
- Proficiency in data analytics languages like Python, Java, Scala, etc.
- Knowledge of bid data technologies like Hadoop, Spark and distributed computing frameworks to manage large scale data processing
- Proficient in using version control systems like Git for managing code and configurations
Want to take your career to the next level? Apply today.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Architecture Big Data Computer Science Data Analytics Data governance Data pipelines Data quality Data strategy Docker Engineering ETL Finance Git Hadoop Java Kubernetes Machine Learning Mathematics NoSQL Pipelines Power BI Privacy Python RDBMS Scala Security Snowflake Spark SQL Statistics Tableau Talend Unstructured data
Perks/benefits: Career development Flex hours
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Manager jobs
- Open Data Science Manager jobs
- Open Principal Data Engineer jobs
- Open Marketing Data Analyst jobs
- Open Power BI Developer jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Business Data Analyst jobs
- Open Data Analytics Engineer jobs
- Open Data Scientist II jobs
- Open Business Intelligence Developer jobs
- Open Product Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Sr Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Senior Data Architect jobs
- Open Sr. Data Scientist jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Research Scientist jobs
- Open Data Quality Analyst jobs
- Open Azure Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Junior Data Engineer jobs
- Open GCP-related jobs
- Open Data quality-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Business Intelligence-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Deep Learning-related jobs
- Open PhD-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open NLP-related jobs
- Open PyTorch-related jobs
- Open TensorFlow-related jobs
- Open APIs-related jobs
- Open LLMs-related jobs
- Open Consulting-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open Snowflake-related jobs
- Open Hadoop-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Databricks-related jobs
- Open Airflow-related jobs