ML Engineer — LLM Evaluation
San Francisco, CA
Dynamo AI
Dynamo AI offers end-to-end AI Performance, Security, and Compliance solutions for delivering Enterprise-grade Generative AI.
At Dynamo AI, we believe that LLMs must be developed with safety, privacy, and real-world responsibility in mind. Our ML team comes from a culture of academic research driven to democratize AI advancements responsibly. By operating at the intersection of ML research and industry applications, our team empowers Fortune 500 companies’ adoption of frontier research for their next generation of LLM products. Join us if you:• Wish to work on the premier platform for private and personalized LLMs. We provide the fastest end to end solution to deploy research in the real world with our fast-paced team of ML Ph.D.’s and builders, free of Big Tech / academic bureaucracy and constraints.• Are excited at the idea of democratizing state-of-the-art research on safe and responsible AI.• Are motivated to work at a 2023 CB Insights Top 100 AI Startup and see your impact on end customers in the timeframe of weeks not years.• Care about building a platform to empower fair, unbiased, and responsible development of LLMs and don’t accept the status quo of sacrificing user privacy for the sake of ML advancement.
Salary for this position may vary based on several factors, including the candidate's experience, expertise, and the geographic location of the role. Compensation is determined to ensure competitiveness and equity, reflecting the cost of living in different regions and the specific skills and qualifications of the candidate.
Responsibilities
- Own LLM evaluation processes and methods with a focus on generating benchmarks representative of real-world usage and safety vulnerabilities.
- Generate high quality synthetic data, curate labels, and conduct rigorous benchmarking.
- Deliver robust, scalable, and reproducible production code.
- Push the envelope by developing methods for benchmarking that revamps how we assess the best LLMs for harmlessness and helpfulness. Your research will directly empower our customers to more feasibly deploy safe and responsible LLMs.
- Co-author papers, patents, and presentations with our research team by integrating other members’ work with your vertical.
Qualifications
- Domain knowledge in LLM evaluation and data curation techniques.
- Extensive experience in designing and implementing LLM benchmarking, extending previous methods. Comfortability with leading end-to-end projects.
- Adaptability and flexibility. In both the academic and startup world, a new finding in the community may necessitate an abrupt shift in focus. You must be able to learn, implement, and extend state-of-the-art research.
- Preferred: past research or projects in benchmarking LLMs.
Salary for this position may vary based on several factors, including the candidate's experience, expertise, and the geographic location of the role. Compensation is determined to ensure competitiveness and equity, reflecting the cost of living in different regions and the specific skills and qualifications of the candidate.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
14
2
0
Categories:
Engineering Jobs
Machine Learning Jobs
Tags: LLMs Machine Learning Privacy Research Responsible AI
Perks/benefits: Equity / stock options Startup environment
Region:
North America
Country:
United States
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Junior Data Analyst jobs
- Open Data Engineer II jobs
- Open Principal Data Scientist jobs
- Open Business Data Analyst jobs
- Open Data Scientist II jobs
- Open Sr Data Engineer jobs
- Open BI Analyst jobs
- Open Business Intelligence Engineer jobs
- Open Sr. Data Scientist jobs
- Open Data Science Intern jobs
- Open Senior Business Intelligence Analyst jobs
- Open Software Engineer, Machine Learning jobs
- Open Lead Data Analyst jobs
- Open Azure Data Engineer jobs
- Open Junior Data Scientist jobs
- Open MLOps Engineer jobs
- Open Manager, Data Engineering jobs
- Open Marketing Data Analyst jobs
- Open Data Analytics Engineer jobs
- Open Data Engineer III jobs
- Open Junior Data Engineer jobs
- Open Data Engineering Manager jobs
- Open Data Analyst II jobs
- Open Product Data Analyst jobs
- Open Data management-related jobs
- Open Data quality-related jobs
- Open Privacy-related jobs
- Open Excel-related jobs
- Open ML models-related jobs
- Open Data pipelines-related jobs
- Open APIs-related jobs
- Open PhD-related jobs
- Open PyTorch-related jobs
- Open Finance-related jobs
- Open LLMs-related jobs
- Open Data visualization-related jobs
- Open TensorFlow-related jobs
- Open Deep Learning-related jobs
- Open Consulting-related jobs
- Open Business Intelligence-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Data governance-related jobs
- Open DevOps-related jobs
- Open Kubernetes-related jobs
- Open Git-related jobs
- Open Docker-related jobs
- Open Hadoop-related jobs