MTS: Data Infrastructure
San Francisco
Essential AI’s mission is to deepen the partnership between humans and computers, unlocking collaborative capabilities that far exceed what could be achieved today. We believe that building delightful end-user experiences requires innovating across the stack - from the UX all the way down to models that achieve the best user value per FLOP.
We believe that a small, focused team of motivated individuals can create outsized breakthroughs. We are building a world-class multi-disciplinary team who are excited to solve hard real-world AI problems. We are well-capitalized and supported by March Capital and Thrive Capital, with participation from AMD, Franklin Venture Partners, Google, KB Investment, NVIDIA.
The Role
The Data Infrastructure Engineer will design, implement, and optimize a scalable infrastructure to prepare the data that powers our AI training. This infrastructure must be reliable and capable of efficiently processing petabytes of data. You will collaborate closely with the data research team and data crawling team when designing this system.
What you will be working on
Building petabyte-scale, high-throughput data processing systems for preparing and curating datasets for AI training.
Orchestrating workloads across large clusters; Architecting and maintaining distributed computing environments.
Working directly with our data research team on implementing new methods of data preparation.
Troubleshooting and resolving infrastructure-related issues in a timely manner.
What we are looking for
Minimum of 3 years of experience in data-intensive applications and software development.
Proficient with Kubernetes & containerization and with building cloud services using providers like AWS, GCP etc.
Ability to write, debug and optimize distributed systems and understanding of data orchestration and automation tools (or strong willingness to learn)
Proficient in high performance programming languages like Go or Rust or C++.
You have previous experience in creating and maintaining infrastructure for processing datasets for ML model training and/or serving
We encourage you to apply for this position even if you don’t check all of the above requirements but want to spend time pushing on these techniques.
We are based in-person in SF. We offer relocation assistance to new employees.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: AWS Distributed Systems GCP Kubernetes Machine Learning Model training Research Rust UX
Perks/benefits: Relocation support
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Data Manager jobs
- Open Principal Data Engineer jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Sr Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Business Data Analyst jobs
- Open Data Analyst Intern jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Manager, Data Engineering jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Business Intelligence-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Generative AI-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Docker-related jobs