Data Engineer Lead

About Ampol Trading & Shipping

Our purpose at Ampol is to power better journeys today and tomorrow. Our Trading and Shipping team is key to achieving this, not only supplying our customer’s needs today, but evolving as the energy transition takes place to supply our customers well into the future.

We take pride in our people and value innovation, sustainability, and customer satisfaction. If you’re a professional committed to excellence, Ampol is looking for you.

About the role

This position will be responsible for working with large and disparate data sets, internal and external, quantitative, and qualitative, structured and unstructured to support the management of data across the entire data lifecycle. This role requires a combination of technical expertise, leadership skills and strong focus on data governance practices to ensure the successful delivery of the Data Product Roadmap. 

The data engineer brings expert understanding of data and modern data processing technologies to prepare and deliver data assets which support high priority projects.

This position is a part of the Data Product Team, which is responsible to delivering on the prioritised T&S Data roadmap to meet strategic objectives. aligned to the Trading & Shipping organisation. 

Key responsibilities

Accountabilities

• Working with the key stakeholders to define data requirements for projects
• Perform data acquisition/ingestion from internal and external sources, including both structured and unstructured data sources (incl. multi-dimension data, text/documents, geospatial data and graph)
• Performing descriptive analytics and data visualisation to share data insights or validate data with stakeholders. Prepare data quality reports
• Develop data transformations and modelling to prepare data sets for consumption for various end uses including reporting or advanced analytics.
• Develop and maintain productionised data outputs/products to support ongoing uses
• Working in a highly agile and flexible environment to deliver outputs in short time frames
• Document technical specifications of data pipelines and technical products delivered
• Develop working understanding of Ampol systems, data, and uses, as a data SME
• Data Strategy and Planning- Continuously develop and enhance Ampol data strategy and architecture and roadmap to ensure alignment to business requirements and overall company vision.
• Data Exploration and Analysis- Explore data to identify patterns, trends and insights. Perform quantitative and qualitative analysis to answer specific business questions and to gain a deeper understanding of data.
• Data Pipeline Development- Build and maintain data pipelines to move and transform data from various sources into Snowflake with assurance that data ingestion is reliable, scalable and optimized for performance.
• Data Modelling and Design- With requirements from business, design and implement data models within Snowflake, including schema design, table structures and data partitioning to support high performance, low latency querying and analytics.
• Data Quality and Integrity- Enforce data quality checks and validation during data ingestion process to ensure accuracy and integrity of data in Snowflake.
• Support Continuous Improvement in Data Product- Support the Commercial Analytics in advancing continuous improvement in the data space, including team engagement to generate awareness of initiatives and track progress.
• Monitoring and Maintenance- Proactively monitor data pipelines and the overall health of Snowflake data infrastructure to identify and resolve issues.
• Automation and Orchestration- Automate data workflows and orchestrate data processes to minimize manual intervention and improve operational efficiency.
• Collaboration with Data Users- Work closely with data analysts, data engineers, and business users to understand their data requirements and provide necessary data support.

Capabilities

• Big Data Engineering – familiarity in designing and developing data processes on cloud based big data platforms
• Data Quality – experience in managing data quality controls to support the completeness and accuracy of data
• Strong understanding of data structures and designing conceptual, logical and physical data models
• Understanding of Information Management, Data Governance, Data Security and Privacy concepts (covered by respective Ampol policies)

Qualifications & experience

• Tertiary qualification in Business, IT, Engineering, Finance or or related discipline or relevant operational experience.
• Project 10+ years of technology and commercial delivery experience with 3 years in a leadership role
• Proficient in digital delivery, technologies and tools
• 3 to 7 year’s downstream oil industry and/or petroleum trading companies in areas such as Supply planning, Supply operations, line finance roles, corporate planning and strategy, and distribution and sales leadership positions.
• Application of agile project methodology

Skills & Experience

• Tertiary qualifications in Computer Science or similar.
• Qualifications or experience in mathematics or statistics highly regarded
• 2+ years of industry experience working as a data engineer to develop data products using SQL, Python, and Spark
• Data modelling experience including ETL
• Data quality management
• Experience with a range of Data platforms (Hadoop, Spark, SQL Databases)  
• Experience performing descriptive analytics and developing data visualisation (Power BI, Tableau, Shiny, etc)
• Experience working to support advanced analytics products is desirable

Knowledge & Skills

• Expertise with SQL, Python, and Spark
• Understanding of git flow and relevant code management principles
• Expertise in establishing and monitoring data pipelines and workflows
• Expertise working with varied Relational Databases
• Tertiary qualification in Data Analytics or related discipline
• Project 3 to 7 years of data engineering in Snowflake
• 3 to 7 year’s downstream oil industry and/or petroleum trading companies in areas such as Supply planning, Supply operations, line finance roles, corporate planning and strategy, and distribution and sales leadership positions.
• Proficiency in programming languages like SQL to work with database sets and data manipulation.
• Solid understanding of database management systems, data storage, data indexing and query optimization for efficient data handling.
• Proficient in data modelling techniques and design patterns to help create efficient and scalable data architectures.
• Effective communication skills and has ability to collaborate with cross functional teams to understand data requirements and deliver valuable insights.
• Understand data quality principles and data governance best practices.

Desirable:

• Working knowledge and understanding of advanced analytic techniques (statistical and predictive modelling and machine learning)
• Experience working with Snowflake and/or Talend
• Understanding of graph or noSQL technologies
• Familiarity with Ampol business domains
• Experience with Docker, Kubernetes
• Proficiency in data analytics languages like Python, Java, Scala, etc.
• Knowledge of bid data technologies like Hadoop, Spark and distributed computing frameworks to manage large scale data processing
• Proficient in using version control systems like Git for managing code and configurations

Want to take your career to the next level? Apply today.