Data Engineer (Hybrid)

Company Description

Fastmarkets is an independent commodity pricing and information organisation with over 600 staff. We are fuelled by values that bring us all together and are united by a collective passion to make a difference. We are supported by a working model that is based on a hybrid approach that allows each of us to balance home and office working while encouraging effective collaboration and accountability.

Fastmarkets is the most trusted Price Reporting Agency (PRA) in the metals, mining, agriculture, forest products, and energy transition markets. Our price data, forecasts, and market analyses give our customers strategic advantage in complex, volatile, often opaque markets. Our events provide forums for market participants to come together, trade, and talk through the critical issues of our times.

We are making a difference. Fastmarkets works to create transparent commodity markets through our reliable and trusted price data and market intelligence. The markets we serve are central to how economies work and are essential for our lives. By doing this, Fastmarkets helps our customers build a more sustainable world. We are built on a 130-year foundation while bringing a digital platform to market, combining the currency of trust with the power of digital.

Most importantly, we are fuelled by our teams that, across the globe, make the amazing possible. With a deep sense of purpose and belonging, we are building an extraordinary future – together. If you would like to help build the future, we hope you will join us on our journey.

Fastmarkets is owned by global private equity firm Astorg, a specialist investor in healthcare, software, technology, business services and technology-based industrial companies.

Job Description

Fastmarkets Data team is a new and dynamic function supporting the business in developing and delivering our data strategy through data, insights, and business intelligence. Highly valued by the business we are working hard to transform data function and ways of working.

Reporting directly into ambitious Data Architect (Data Platform Manager) we are looking for the right candidate to model and build our data pipelines and transformations. This is a fantastic opportunity to join us at the start of our journey to bring Data, BI, Analytics, and Insights together into a high performing workstream along with delivering transformational change. You will have the opportunity to work with many of the latest technologies and frameworks – with input to help deliver & adapt processes that will enable a true “DataOps” environment and development experience.

The Data Engineer will ensure that data is cleansed, tested, and delivered in accordance with Fastmarkets and industry best practice. In addition, the data engineer will enable monitoring via the observability platform and that notifications and alerts are appropriately configured.

We also seek to leverage the latest AI (Artificial Intelligence) tools to assist the data-engineering team, in tasks such as documentation and formatting of code.

The role requires a high standard of expertise in SQL and Python as well as specialist knowledge of modern approaches delivering data projects using a code-first approach.

Principal Accountabilities

• Design, develop, and maintain scalable and efficient data pipelines from a wide variety of sources
• Use DBT to transform the data through the various layers (Cleansed, Conformed, Presentation etc)
• Ability to write custom connectors (Python) and leverage “out-of-the-box” data-loading tools
• Operationalise enterprise data model by curating appropriate data models to service reporting, analytics, and data science use cases
• Embed real-time, automated data quality checks, validations, exceptional handling and alerting across all data pipelines
• Data Orchestration & Observability Platforms
• Manage CI/CD workflows, using GitHub, GitHub Actions and Kubernetes.
• Using Zero-Copy cloning and containerized “on-demand” development environments.
• Implementing and reviewing of RBAC policies
• Embrace DataOps & “Code first” approach
• Identify and promote best practices in data engineering. Ability to recommend improvements to existing processes and systems
• Excellent problem-solving skills and ability to embrace change
• Effective communication and collaboration skills
• Natural self-starter, with enthusiasm for learning and research

Qualifications

We recruit talented, dynamic people with diverse backgrounds and experiences, all united by a belief in our mission to provide the world’s leading and most trusted price reporting, events, and intelligence service for the markets we serve. We’re proud to be an equal opportunities employer and are committed to creating a fully inclusive workplace, where everyone feels able to participate and contribute meaningfully.

If you are open-minded, curious, resilient, solutions-oriented and committed to promoting equality, then read on...

KNOWLEDGE, EXPERIENCE AND SKILLS

We are looking for an individual who is highly motivated, driven, and have a passion to be part of a fast-paced, successful team. Being a strong team player is also important as well as someone who is happy to work flexibly.

• Advanced SQL Expertise – CTE’s, Stored Procs, Query Tuning – across more than one “flavour” of SQL (i.e. Snowflake / MSSQL / Databricks etc)
• DBT – Data Build Tool & Jinja
• Extensive hands-on experience working with cloud-based data warehousing including Snowflake, ELT tools such as Fivetran, Talend, Informatica and Python Libraries such as Snowpark, PySpark, Pandas, GreatExpectations  etc
• API integration & JSON parsing/transformation
• Event Streaming (Azure Service Bus, Kafka, SQS, RabbitMQ etc)
• Python data pipelines & Orchestration platforms (i.e. Dagster, Apache Airflow or Prefect)
• Git, (Branching, Pull Requests, Merge Conflict resolution)
• CI/CD principles and experience (GitHub actions, MS DevOps etc)
• Cloud Services – Ability to identify and configure suitable cloud services and resources (Azure Preferred)
• Kubernetes (nice to have)
• IaaS – (nice to have)
• Solid understanding of data modelling approaches, Note: Data-Vault 2.0 desirable (3NF, Star, Kimball/Inmon, OBT, Data-Vault etc)
• Excellent analytical and problem-solving skills with experience working with large and complex data sets
• Exposure to CRM systems, such as Salesforce and digital marketing solutions, such as Marketo
• Strong analytical, problem-solving and communication skills with attention to detail

QUALIFICATIONS

• Degree in related technology field OR 3+ years’ experience in similar role with professional certifications.
• Snowflake – Ideally with SnowPro Certification
• Can demonstrate experience working on a project that delivered transformational change, or exceptional business value – or personal technology related project/hobby that needing significant research, learning & dedication to achieve the outcome.

If you're excited about the role but your experience, skills or qualifications don't perfectly align, we encourage you to apply anyway.

Additional Information

Our Values

Fastmarkets people come from all different walks of life. It’s this mix of brilliant personalities, experiences and insights that gives us that warm, open, and friendly culture you can feel as soon as you meet us. But however wonderfully different we all are, there are six things we all have in common – and they form our Fastmarkets values.

Created by our own employees to reflect some of the personal traits that Fastmarkets people have, our values are key to what makes our culture unique. They reflect who each of us are and they're embedded in everything we do. Our values are:

• METRICS DRIVEN. We use insights to improve our customers’ experience and our business performance
   
• ACCOUNTABLE. We are accountable to ourselves and those we work with: we keep our promises and get things done
   
• GROWTH MINDSET. This value enables us to be nimble to the changing realities and operate with a sense of urgency
   
• INCLUSIVE. We are inclusive and respectful, celebrating each of us and giving everyone a deep sense of belonging with the desire to bring their best self to work every day.
   
• CUSTOMER CENTRIC. We are customer-centric in all that we do
   
• COLLABORATIVE. We are collaborative, able to work across teams and capitalise on the diversity of intellect, perspectives, and experiences.

You’ve read a little about us – now it’s over to you!

If you like what you’ve read so far and think you can see yourself as a Fastmarkets person, it’s time to fill in your application form. This form is an important part of the selection process: it’s used to determine whether or not you’ll be chosen to have an interview and acts as a basis for the questions we’ll ask you on the day.

It’s vital that you try to capture all the relevant information we have asked for on the form so we can get a good feel for who you are and why you’re great.