Data Engineer
London, UK
Octopus Energy
Octopus customers pay less: Cheaper than price-cap prices, rewards for using less when the grid is stressed, and smart tariff savings for your home, car or batteryAbout our Data Platform Team
At Kraken we've developed a data platform that that is used by Octopus Energy and our other clients retail energy businesses around the world. The platform empowers users with self-service data analytics and automates our data processing workflows, from simple ETL jobs to ML training and prediction.
The data platform team works across the whole customer domain on anything from natural language understanding of our customer communications to processing billions of smart meter readings every day to support clients in creating customised and market leading smart energy tariffs.
As the volume, scope and geographical range of our data offerings rapidly expand, we're looking for an experienced data engineer to join the team to help us build and maintain our platform, pipelines and data sources.
We're passionate about building great technology to change the way customers use, and think about, energy for the good of the planet. This is a fantastic opportunity to work with us on data problems that genuinely move us closer to Net Zero, and support the energy transition.
What you'll do...
- Build new data sources and data pipelines that deliver key data and insights to the business
- Work closely with the data science and analytics teams to maintain and develop our central data models in dbt
- Build and maintain testing and documentation frameworks for our data sources
- Work with the business to scope and deliver new data engineering projects and requirements
- Maintain and build on our existing data infrastructure and tools
- Support the internationalisation of our data infrastructure as we continue to grow globally
What you'll have...
- First and foremost, we want our data engineers to be great software engineers with a passion for writing high quality code. It would be helpful to have experience/expertise in the following (in rough priority order):
- Python
- SQL
- Spark
- Experience modelling data for analytics - ideally experience using dbt as a modelling tool
- Experience in assuring data quality
- Experience deploying data services in a cloud environment (ideally AWS)
- The projects will be varied and we’re looking for someone who can work autonomously and proactively to scope problems and solve and deliver pragmatic solutions
Our Data Platform Stack...
- Databricks our datalake approach
- Spark for data processing
- Python as our main programming language
- Kubernetes for data services and task orchestration
- dbt for data modelling
- Airflow purely for job scheduling and tracking
- Circle CI for continuous deployment
- Parquet and Delta file formats on S3 for data lake storage
- SparkSQL for analytics
- Streamlit for data applications
Why else you'll love it here
- Wondering what the salary for this role is? Just ask us! On a call with one of our recruiters it's something we always cover as we genuinely want to match your experience with the correct salary. The reason why we don't advertise is because we honestly have a degree of flexibility and would never want salary to be a reason why someone doesn't apply to Octopus - what's more important to us is finding the right octofit!
- Octopus Energy is a unique culture. An organisation where people learn, decide, and build quicker. Where people work with autonomy, alongside a wide range of amazing co-owners, on projects that break new ground. We want your hard work to be rewarded with perks you actually care about! We won best company to work for in 2022, on Glassdoor we we're voted 50 best places to work in 2022 and our Group CEO, Greg has recorded a podcast about our culture and how we empower our people
- Visit our perks hub - Octopus Employee Benefits
Above all, we’re looking for someone who thrives on providing alternative opinions, challenging those around them and being challenged.
If this sounds like you then we'd love to hear from you.
Studies have shown that some groups of people, like women, are less likely to apply to a role unless they meet 100% of the job requirements. Whoever you are, if you like one of our jobs, we encourage you to apply as you might just be the candidate we hire. Across Octopus, we're looking for genuinely decent people who are honest and empathetic. Our people are our strongest asset and the unique skills and perspectives people bring to the team are the driving force of our success. As an equal opportunity employer, we do not discriminate on the basis of any protected attribute. Our commitment is to provide equal opportunities, an inclusive work environment, and fairness for everyone.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow AWS Data Analytics Databricks Data pipelines Data quality dbt Engineering ETL Kubernetes Machine Learning Parquet Pipelines Python Spark SQL Streamlit Testing
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Lead Data Analyst jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Data Engineer II jobs
- Open Data Manager jobs
- Open Principal Data Engineer jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Sr Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Business Data Analyst jobs
- Open Data Analyst Intern jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Generative AI-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs