We are looking for a Principal Engineer with deep ML engineering expertise to lead the ML and science engineering effort across the AI Platforms organization at AWS. This is a cross-cutting leadership role spanning the full breadth of our ML development platform: data preparation, model evaluation, model deployment and customization, and agentic AI development experience.
In this role, you will be responsible for building production-grade 1P agents, designing comprehensive evaluation frameworks (including LLM-as-Judge), creating recommendation systems for model benchmarking and selection, and driving the science-engineering interface to deliver superior developer experiences at AWS scale. You will work closely with world-class science teams to translate innovative research into production systems and establish the technical strategy for ML engineering across the organization.
You are not just an architect — you are a hands-on builder who can prototype alongside scientists, write production code, and review designs across multiple teams. You bring a rare combination of ML depth, systems thinking, and the ability to influence without authority at a global scale.
Key job responsibilities
• Design and build production-grade 1P agent architectures including memory management, prompt optimization, tool use, and agentic orchestration systems
• Define and own the evaluation strategy for AI Platforms, including LLM-as-Judge frameworks, automated benchmarking, and model quality assessment pipelines
• Build ML-driven recommendation systems for model benchmarking, selection, deployment, and customization from Model Hub
• Establish data quality evaluation pipelines and synthetic data generation infrastructure to support model training and fine-tuning at scale
• Drive requirements and technical roadmap with the science team; translate research prototypes into engineering specifications and production systems
• Define the ML engineering technical strategy across AI Platforms, establishing best practices for agent development, model evaluation, and science-to-production pipelines
• Mentor senior engineers across the organization; conduct design reviews, technical deep-dives, and bar-raising interviews
• Present technical strategy and roadmap to VP-level stakeholders; represent AI Platforms ML engineering in cross-organizational technical forums
