Job ID: 3179188 | Amazon.com Services LLC
This is currently a 12 month temporary contract opportunity with the possibility to extend to 24 months based on business needs. The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, and to build industry‑leading technology with Large Language Models (LLMs) and multimodal systems.
Key job responsibilities
As part of the AGI team, an Applied Scientist will collaborate closely with the core scientist team developing Amazon Nova models. The role will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. Responsibilities include designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies; performing expert‑level manual audits and meta‑audits to evaluate auditor performance; providing targeted coaching to uplift overall quality; developing and maintaining LLM‑as‑a‑Judge systems; designing judge architectures and evaluation rubrics; building machine learning models for automated quality assessment; setting up data collection workflow configuration; communicating quality feedback to stakeholders; and enhancing customer experiences through high‑quality training and evaluation data.
A day in the life
An Applied Scientist with the AGI team will support quality solution design, conduct root‑cause analysis on data quality issues, research new auditing methodologies, and find innovative ways to optimize data quality while setting examples for the team on quality assurance best practices and standards. The role also involves working closely with engineers, domain experts, and vendor teams to implement quality strategies and automated judging systems.
Basic Qualifications
- Master's degree in engineering, statistics, computer science, mathematics, or a related quantitative field
- 2+ years of machine learning, statistical modeling, data mining, and analytics techniques experience
- 3+ years of programming in Java, C++, Python or related language experience
- 2+ years of building machine learning models or developing algorithms for business application experience
Preferred Qualifications
- Ph.D. in engineering, technology, computer science, machine learning, robotics, operations research, statistics, mathematics, or equivalent quantitative field
- Publications at top‑tier peer‑reviewed conferences or journals
- 4+ years of solving business problems through machine learning, data mining, and statistical algorithms experience
- Experience with machine learning and large language model fundamentals, including architecture, training/inference lifecycles, and optimization of model execution, or experience debugging, profiling, and implementing best software engineering practices in large‑scale systems
Amazon is an equal‑opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information.
The starting pay for this position is $136,000.00 – $184,000.00 USD annually, which varies by location. Starting Day 1 of employment, Amazon offers EAP, Mental Health Support, Medical Advice Line, and 401(k) matching. Learn more about our benefits at https://hiring.amazon.com/why-amazon/benefits.
USA, MA, Boston – $136,000.00 – $184,000.00 USD annually
USA, WA, Bellevue – $136,000.00 – $184,000.00 USD annually
#J-18808-Ljbffr