Interested in building the future of intelligent AI agents? The Agentic AI organization at AWS is on a mission to empower developers and enterprises to create, evaluate, and deploy autonomous agents that can reason, plan, and act using state-of-the-art GenAI systems. Our Evaluations team plays a central role in this mission by enabling rigorous, scalable, and automated evaluation of GenAI and agentic systems.
As a Software Dev Engineer on the Evaluations & Optimization team, you will build state-of-the-art systems that help customers assess the quality, performance, and reliability of Agents and build state of the art solutions for agents optimization. You will define and deliver foundational infrastructure and developer tooling that powers evaluation workflows for some of the world’s most advanced GenAI and Agentic systems.
Our work spans deep learning, large-scale distributed systems, and evaluation science workflows. We work closely with applied scientists, product managers, and UX designers to deliver services that are robust, extensible, and easy to use for developers and enterprise customers alike.
Key job responsibilities
• Design and implement core runtime infrastructure components including control plane APIs, data plane execution environments, and deployment orchestration systems
• Develop monitoring, observability, and operational tooling to ensure production reliability and enable rapid incident response
• Collaborate with AWS Lambda, Bedrock, and open-source framework teams (LangChain, LangGraph, Strands) to integrate and optimize agent capabilities
• Contribute to technical strategy and architecture decisions for session management, identity/permissions, cross-account access, and security integrations
About the team
Agentic AI drives innovation at the forefront of artificial intelligence, enabling customers to transform their businesses through cutting-edge AI solutions. We build and deliver the foundational AI services that power the future of cloud computing, helping organizations harness the potential of AI to solve their most complex challenges. Join our dynamic team of AI/ML practitioners, applied scientists, software engineers, and solution architects who work backwards from customer needs to create agentic technologies. If you're passionate about shaping the future of AI while making a meaningful impact for customers worldwide, we want to hear from you.
