Amazon Q Business is an AI assistant powered by generative technology. It provides capabilities such as answering queries, summarizing information, generating content, and executing tasks based on enterprise data.
We are seeking a Language Data Scientist II to join our data team. Our mission is to engineer high-quality datasets that are essential to the success of Amazon Q Business. From human evaluations and Responsible AI safeguards to Retrieval-Augmented Generation and beyond, our work ensures that Generative AI is enterprise-ready, safe, and effective for users.
As part of our diverse team—including language engineers, linguists, data scientists, data engineers, and program managers—you will collaborate closely with science, engineering, and product teams. We are driven by customer obsession and a commitment to excellence.
In this role, you will leverage data-centric AI principles to assess the impact of data on model performance and the broader machine learning pipeline. You will apply Generative AI techniques to evaluate how well our data represents human language and conduct experiments to measure downstream interactions.
Key job responsibilities
* oversee end-to-end evaluation data pipeline and propose evaluation metrics and methods
* incorporate your knowledge of linguistic fundamentals, NLU, NLP to the data pipeline
* process and analyze diverse media formats including audio recordings, video, images and text
* perform statistical analysis of the data
* write intuitive data generation & annotation guidelines
* write advanced and nuanced prompts to optimize LLM outputs
* write python scripts for data wrangling
* automate repetitive workflows and improve existing processes
* perform background research and vet available public datasets on topics such as long text retrieval, text generation, summarization, question-answering, and reasoning
* leverage and integrate AWS services to optimize data collection workflows
* collaborate with scientists, engineers, and product managers in defining data quality metrics and guidelines.
* lead dive deep sessions with data annotators
