The AGI Data Services organization is responsible for procuring data needed for building and validating large-language models. The vision of the AGI-DS Data Quality org is to accurately measure the overall quality and specific utility of every unit of data distributed by the AGI-DS organization to our customers; to root cause the reason that quality and utility are low and propose solutions for how to solve; and to manage quality control relationships with delivery channels.
Today, AGI Data Services produces tens of millions of units of human-created data that is used to train and fine-tune large language models. These data must be evaluated based on whether they were generated following the conventions defined by the workflow, on dimensions such as complexity, correctness, helpfulness and harmfulness. Filtering out high quality data from low quality is of paramount importance, as it maximizes our chances of improving performance of the model under test.
To achieve this objective, the Data Quality organization is responsible for auditing individual units of data, as well as building an automated system using a large language model to predict and filter high quality units of data.
This TPM role’s scope includes defining strategy, processes and requirements, and leading cross functional development teams from design through delivery. You will be accountable for driving the entire product lifecycle, from platform product definition through specification, coding, quality assurance and launch to the world. Most importantly, you will influence engineering teams to build the right solutions. You will be also responsible for creating leverage by ensuring capabilities are reusable and scalable to world-wide legislative changes. This position involves regular communication with senior management on status, risks and change control. Cross-team coordination, project and program management and executive presentation skills are essential.
Key job responsibilities
The Sr. Technical Program Manager will interact with product management, program management, cross partner teams, multiple stakeholders and leadership to deliver complex projects and programs. The person will anticipate bottlenecks, provide escalation management, make trade-off decisions, and balance business needs versus technical constraints.
The person should have technical skills to provide solutions, design, build, and deliver it. Successful candidates will have a strong technical background, be detail driven and have excellent problem solving abilities. You will have a deep understanding of product design and implementation details and will successfully represent the technical team to senior leadership and customers. Your technical expertise and operational excellence will influence the team’s decisions and drive secure and robust solutions, which have great customer experience. They are responsible for bridging scientists, technology, process and operations for to achieve the right results. This position defines standards and measurements, identifies the inputs that drive the quality of Ground Truth, and drives initiatives across a variety of teams to achieve these results.
A day in the life
- Collaborate with internal partners to understand their business priorities, communicate outcomes and define roadmaps.
- Define metrics and performance indicators and establish mechanisms to drive them.
- Dive deep into anecdotes and data to find patterns and root causes of problems.
- Dive deep into science, technology and process to identify problems and create solutions. Tirelessly advocate for improvements to systems and processes that will benefit our product teams / stakeholders and business partners.
- Engage with ML Data operations teams to root cause issues, and design solutions.
