We are looking for a Forward Deployed Machine Learning Engineer with 3+ years of experience to be the first MLE dedicated to the Benchmarks and Evaluations vertical at a fast-scaling, venture-backed AI data infrastructure company. You will work directly with the GM and researchers to build the technical foundation for AI model evaluation, a critical capability for the business.
What will you be doing?
Partner with the GM and early customers to define, design, and build benchmarks and evals across different domains and modalities.
Build and own the backend infrastructure: data pipelines, execution environments, storage, and orchestration.
Stand up sandboxed environments for agentic evals requiring tools, code execution, or multi-step tasks.
Own the engineering portion of customer engagements end to end.
Identify repeatable eval patterns and infrastructure gaps to drive product opportunities.
- 4+ years of engineering experience with hands-on ML model evaluation work
- Prior ownership of backend and infrastructure (data pipelines, execution environments)
- Experience building benchmarks, evals, or human data pipelines for LLMs (strongly preferred)
- High ambiguity tolerance and a strong bias to action in fast-moving environments
- Strong written communication skills for customer-facing and cross-functional work
- Experience deploying end-to-end ML evaluation or benchmark systems to production for foundation models against tight customer deadlines
- Proficiency in ML eval frameworks and benchmark design (e.g. LLM-as-judge)
- 3-8 years of experience in machine learning engineering, eval and benchmark systems, and end-to-end ownership in fast-moving or founding-stage environments
- Customer-facing engineering experience managing enterprise customers
- Experience building data pipelines for large-scale data
- Ability to scope and execute ambiguous technical problems independently, from feasibility through delivery
- Experience interfacing with AI researchers or foundation labs
- Published work or open-source contributions in ML evals or benchmarks preferred
- Degree in Computer Science, Physics, or related technical field
- Strong skills in Python, LLMs, data pipelines, agentic systems, code execution sandboxes, backend infrastructure, and orchestration