Machine Learning Infrastructure Engineer - Generative AI Team
Madrid
Adyen
End-to-end payments, data, and financial management in one solution. Meet the financial technology platform that helps you realize your ambitions faster.This is Adyen
Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice. At Adyen, everything we do is engineered for ambition.
For our teams, we create an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers. We are motivated individuals who tackle unique technical challenges at scale and solve them as a team. Together, we deliver innovative and ethical solutions that help businesses achieve their ambitions faster.
Adyen seeks a Machine Learning Infrastructure Engineer to join our Generative AI team in Madrid. The team focuses on enhancing efficiency among different internal operational verticals through AI automation. We specialize in creating a platform for Large Language Models (LLMs) and its downstream applications using Open Source software like HuggingFace or Langchain, and models such as Falcon, Llama or Mixtral as well as OpenAI or other proprietary LLMs.
Key responsibilities:
- Collaborate with the team to deploy LLMs for downstream tasks such as ticket routing (text classification), summarization, sentiment analysis, and question-answer retrieval.
- Develop and manage ETL pipelines using PySpark and Airflow for data preprocessing and model training. Organize the orchestration of pipelines for ETL and machine learning training to streamline model development.
- Automate the ML pipeline using MLOps tools and practices and optimize it for scalability and performance.
- Containerize applications and manage the Kubernetes deployments as well as the infrastructure needed to deploy LLMs internally; from GPUs to vector databases and inference components.
- Develop observability best practices for the whole LLM infrastructure and build the internal framework which allows the team to monitor the LLM behavior to ensure their robustness under real life conditions.
- Design and implement APIs or frameworks to facilitate the seamless integration and usage of LLMs within various applications and services.
- Stay up to date with the latest advancements in MLOps tools and practices.
Qualifications:
- 5+ years of professional experience as a Machine Learning Engineer, DevOps or MLOps Engineer showing a clear understanding of the end-to-end machine learning lifecycle
- Strong software development skills, including: version control (e.g. Git and preferable on Gitlab), coding best practices, debugging, unit and integration testing.
- Proficient in Python. Knowledge of PySpark, Airflow, MLflow, Terraform, Docker, Kubernetes, Helm, Kustomize and ArgoCD.
- Proficiency with observability tools, such as: Prometheus, Logsearch, Kibana and Grafana.
- Knowledge of data pipelines and ETL processes to prepare and manage data for ML training and inference. As well as model development and deployment frameworks.
- Solid understanding of DevOps best practices and tools to automate software development and deployment processes, and CI/CD concepts and experience in implementing these practices.
- Ability to diagnose and resolve model performance, scalability, and deployment issues.
- Familiarity with monitoring tools to track model performance, resource utilization, and system health. Experience in logging and error monitoring for ML models and applications.
Desirable additional requirements:
- Experienced with Open-source Machine Learning frameworks like Huggingface Transformers.
- General LLMOps experience is a plus, including model deployment, monitoring, resources, and infrastructure management.
Our Diversity, Equity and Inclusion commitments
Our unique approach is a product of our diverse perspectives. This diversity of backgrounds and cultures is essential in helping us maintain our momentum. Our business and technical challenges are unique, and we need as many different voices as possible to join us in solving them - voices like yours. No matter who you are or where you’re from, we welcome you to be your true self at Adyen.
Studies show that women and members of underrepresented communities apply for jobs only if they meet 100% of the qualifications. Does this sound like you? If so, Adyen encourages you to reconsider and apply. We look forward to your application!
What’s next?
Ensuring a smooth and enjoyable candidate experience is critical for us. We aim to get back to you regarding your application within 5 business days. Our interview process tends to take about 4 weeks to complete, but may fluctuate depending on the role. Learn more about our hiring process here. Don’t be afraid to let us know if you need more flexibility.
This role is based out of our Madrid office. We are an office-first company and value in-person collaboration; we do not offer remote-only roles.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow APIs CI/CD Classification Data pipelines DevOps Docker ETL Generative AI Git GitLab Grafana Helm HuggingFace Kibana Kubernetes LangChain LLaMA LLMOps LLMs Machine Learning MLFlow ML infrastructure ML models MLOps Model deployment Model training OpenAI Open Source Pipelines PySpark Python Terraform Testing Transformers
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Manager jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Science Manager jobs
- Open Principal Data Engineer jobs
- Open Data Engineer II jobs
- Open Sr Data Engineer jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Data Analytics Engineer jobs
- Open Junior Data Scientist jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Business Data Analyst jobs
- Open Data Analyst Intern jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Manager, Data Engineering jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Research Scientist jobs
- Open GCP-related jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open Deep Learning-related jobs
- Open PhD-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open PyTorch-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open CI/CD-related jobs
- Open LLMs-related jobs
- Open Kubernetes-related jobs
- Open Generative AI-related jobs
- Open Data governance-related jobs
- Open Hadoop-related jobs
- Open Airflow-related jobs
- Open Docker-related jobs