Amazon Nova AI Challenge: FAQs

Including details about the 2026 Trusted Software Agent track.
General
What is the Amazon Nova AI Challenge?
The Amazon Nova AI Challenge is a university competition dedicated to accelerating the field of artificial intelligence (AI). It was created to recognize and advance students from around the globe who are shaping the future of artificial intelligence. Student teams are able to work on the latest challenges in the field of AI and build innovative solutions.
How does the Amazon Nova AI Challenge support research?
The Amazon Nova AI Challenge is a testbed for university students to experiment with and advance AI at scale. Participating teams compete to develop innovative and effective solutions to a specific challenge. Teams receive a number of forms of support, including stipends, AWS credits, and consultation and mentoring from the Amazon Nova AI Challenge team.
How do I contact Amazon if I have question about the challenge?
If you can't find an answer to your question, please email: [email protected].
Competition details
What is the goal of this year's challenge?
The goal of the 2026 Amazon Nova AI Challenge: Trusted Software Agents track is to make effective AI systems that remain responsible and safe for all, with a focus on the task domain of agentic systems that assist users in the development of full stack web applications. The ultimate goal of the competition is to identify ways for LLM-enabled systems to handle more complex development tasks while anticipating and mitigating safety risks.
What is in scope for this competition?
The Amazon Nova AI Challenge: Trusted Software Agents track is focused on ensuring LLMs supporting AI-assisted software development remain safe, reliable, and trustworthy. For the 2026 Nova AI Challenge, we will focus on web application development as the coding task domain, with verifiable pass/fail conditions as the primary utility measure for a model developer’s system. We will continue with the “tournament-style” format used in 2025, where ten university teams — five model developer (defense) teams and five red (attack) teams compete in a series of interactive tournaments. There is an increased emphasis on advances in utility (ability to handle more complex development tasks) in parallel with safety (avoiding vulnerabilities and risk of exploits as systems handle more complex tasks) this year. Also, red teams will build systems that detect and exploit vulnerabilities in the applications built by model developer teams. 
How is this Trusted Software Agent track different than the Trusted AI track?
The Amazon Nova AI Challenge: Trusted Software Agents track is a natural extension of the Amazon Nova AI Challenge: Trusted AI track. Both focus on ensuring the safety of coding assistants. Trusted Software Agents adds additional emphasis on advances in coding utility and the domain has expanded from simpler code generation to software engineering tasks such as full stack web development. In the Trusted Software Agents track, the red teams will focus on using LLMs to identify potential vulnerabilities and exploits in applications; whereas the Trusted AI track focused on getting LLMs to generate malicious or vulnerable code.
Why should I participate?
There are multiple benefits of participating in the Amazon Nova AI Challenge: Trusted Software Agents track, including:
  1. Dynamic feedback: Teams will get the opportunity to test their systems against best-in-class competitors. Unlike static benchmarks, the challenge evaluations are dynamic and multi-turn and evolve as both sets of teams refine their systems over the course of the competition.
  2. AWS services: Participating teams will receive training, support, and access to the full suite of AWS services, with monthly AWS credits to support the cost of training and execution of their systems.
  3. IP ownership: Teams retain ownership of their work and associated IP, and are encouraged to publish their research after Amazon’s review.
  4. Stipend: Each team chosen for the Amazon Nova AI Challenge: Trusted Software Agents track will receive sponsorship in the amount of $250K. Funding is intended to support roughly two full-time students and one month of faculty time.
  5. Cash prizes: For model developer teams, the top ranked team will receive $250K and the second ranked team will receive $100K. Red teams will also receive $250K for the top ranked team and $100K for the second ranked team. All cash prizes will be divided equally among the students on the team.
When is the finals event?
The finals event will be held in September 2026. This is a closed-door event and participating teams are not invited to the Finals event.
Can we use other funding to help us participate in this challenge?
Yes, you may use other funding to support your team, subject to the terms described in the Challenge Rules. External funding must be disclosed to Amazon.
Can we publish our research from participation in the Amazon Nova AI Challenge: Trusted Software Agents track?
Yes, in fact publishing research papers as an outcome of your work on the Amazon Nova AI Challenge: Trusted Software Agents track is required for all teams participating in the competition, although teams may not publish any Amazon confidential information, as described in the Challenge Rules. The Amazon Nova AI Challenge requires all teams to submit a technical paper to be included in the challenge proceedings. Your team will not be selected for the finals if your team does not submit a technical paper for inclusion in the proceedings. Papers will be published online at the end of the competition and made publicly available through the Amazon Science website. Teams may also publish research papers in third-party publications and conferences, as long as all papers are provided to Amazon for review and approval at least two weeks before the submission deadline.
How will the tournaments be run?
The Amazon Nova AI Challenge: Trusted Software Agents track will consist of a series of interactive tournaments held from early 2026 through summer 2026. Each tournament will consist of two phases: a build phase focused on coding utility performance and a validation phase focused on application security. 

In the build phase, model developers will interact with user simulators to implement new features for an existing web application. Their success will be determined using a suite of end-to-end test cases. In the validation phase, red team systems will have the opportunity to identify and exploit vulnerabilities in the applications created in the build phase. In addition to building their coding assistants, model developers will also be tasked with building simulations of malicious users, and red teams will be tasked with building simulations of benign users. In the build phase, the coding assistants will interact with both the benign user simulators and the malicious user simulators. The red teams will not have the opportunity to find vulnerabilities in the applications resulting from interaction with their own user simulators, and model developer bots will only be matched up with malicious simulators from other defending teams. At the conclusion of each tournament, teams on each side will be ranked with respect to their performance, and the results will be shared with each team.
How will developer teams be ranked in the tournament?
Model developer teams will be ranked based on a combination of their coding utility performance and the security of the resulting applications. Coding utility performance will measure the number of tasks completed successfully across interactions with multiple user simulators (task success rate). Security performance will measure the number of resulting applications in which the red team bots do not find any vulnerabilities or exploits (defense success rate).
How will red teams be ranked in the tournament?
Red teams will be ranked on a combination of their coding utility performance and their effectiveness at identifying and exploiting vulnerabilities in applications (attack success rate). Coding utility performance will measure the number of tasks completed successfully (task success rate) as their user simulator interacts with each of the model developer teams’ coding assistants. This motivates each red team to ensure their user simulator is realistic and effective. Attack success rate for red teams will be a measure of the number of applications from the build phase in which they successfully identify or exploit vulnerabilities. In the build phase, red teams have an indirect motivation to ensure the security of the applications built through interaction with their user simulator as in doing so they preclude the ability of other red teams to score attack success points on those applications.
Eligibility
Who can apply to participate?
The Amazon Nova AI Challenge: Trusted Software Agents track is open to full-time students (undergraduate or graduate) with some exceptions (see Challenge Rules). Proof of enrollment will be required to participate.
Can I participate if I don’t attend a university?
No. The Amazon Nova AI Challenge: Trusted Software Agents track is open only to full-time enrolled university students.
Do I need to be enrolled in a university program throughout my participation in the competition?
All participating team members must remain full-time students in good standing at their university while participating in the competition.
Do I need to be a certain age?
Participants must be at or above the age of majority in the country, state, province, or jurisdiction of residence at the time of entry.
Can I enroll if a family member is an Amazon employee?
Immediate family members and household members of Amazon employees, directors, and contractors are not eligible to participate. See Challenge Rules for additional restrictions.
Teams
How many teams will be selected to participate?
All applications will be reviewed and evaluated by Amazon. Up to ten teams will be selected to compete in a tournament style competition.
How many team members can our team have?
There is no minimum or maximum number of team members. All team members must be enrolled in their university throughout their participation. All teams will receive a $250,000 grant regardless of how many members are on their team.
Can students from different universities be on the same team?
No. Teams must be composed of students attending the same university.
Can one university have more than one team?
Yes, universities may have more than one team. Multiple teams cannot have the same faculty advisor.
Can I participate on two separate teams?
No. You can only be a part of one team for the duration of the competition.
Can undergraduate and graduate students work together?
Yes, teams may be composed of undergraduate and graduate students.
Do I need a faculty advisor?
All teams must nominate a faculty advisor and include the faculty advisor’s consent in the applications.
Can there by more than one faculty advisor in a team?
Yes, there may be up to three faculty advisors per team.
What is the role of the faculty advisor?
Faculty advisors will advise students on technical direction and be a sounding board for new ideas, similar to a graduate school advisor. They will also act as the official representative from the university for this competition.
Can we add or remove team members during the competition?
During the competition, there will be a period of time during which faculty advisors may request to remove or add members to the team, subject to approval by Amazon. See Challenge Rules for details.
Can we discuss our work with faculty or students who aren’t on our team?
Only team members may work on their systems. However, the faculty advisor and other students and faculty members at your university may provide support and advice to your team and may co-author technical publications and research papers.
Application process
How do we apply to participate in the challenge?
Begin the application via YouNoodle, which will be open only during the application period.
What do we need to apply?
Once you have selected your team members, team leader, and faculty sponsor, you are ready to begin the application process. You may apply to both roles and if you do so Amazon will assign one of the two roles to your team.
Do all team members have to apply?
We prefer that the team leader submit an application on behalf of the whole team. In case a team lead has not been chosen, then any member of the team may submit an application. Each team should only submit one application, and it must include all of your team members’  and Faculty advisors’ information.
Is there an application fee?
There is no application fee.
How will teams be selected to participate?
All applications will be reviewed by a panel of experts from Amazon. Teams will be selected based on the following criteria: (1) the technical merit of the approach; (2) the novelty of the ideas; and (3) an assessment of the team’s ability to execute their plan. Please be sure to provide enough detail in your application to enable evaluation of your proposal.
Grants and prizes
Do we get a grant or other support to participate in the Amazon Nova AI Challenge: Trusted Software Agents track?
Up to ten teams will be sponsored to participate in the competition. Each sponsored team’s university will receive a $250,000 research grant to help fund the team’s participation. In addition each participating team will receive AWS credits to support the development of their system, and support from the Amazon Nova AI Challenge team.
How can the grant be spent?
The grant is intended to support two full-time students for the duration of the competition and one month of the faculty advisor’s salary. No more than 35% of the research grant may be allocated to administrative fees. Entrant Teams will be expected to use the stipend towards any uncovered costs associated with bootcamp and or summit (if any). If your team would like to use the funds in another manner, your faculty advisor must receive approval from Amazon before doing so.
What are the prizes for winning the competition?
From the evaluation at the finals event, the two top ranked model developer teams and top two ranked red teams will receive awards. The two teams placed 1st in each role (i.e., red team and developer team) will receive $250,000 each, and the two teams in 2nd place will receive $100,000 each.
US, CA, San Francisco
Amazon has launched a new research lab in San Francisco to develop foundational capabilities for useful AI agents. We’re enabling practical AI to make our customers more productive, empowered, and fulfilled. In particular, our work combines large language models (LLMs) with reinforcement learning (RL) to solve reasoning, planning, and world modeling in both virtual and physical environments. Our research builds on that of Amazon’s broader AGI organization, which recently introduced Amazon Nova, a new generation of state-of-the-art foundation models (FMs). Our lab is a small, talent-dense team with the resources and scale of Amazon. Each team in the lab has the autonomy to move fast and the long-term commitment to pursue high-risk, high-payoff research. We’re entering an exciting new era where agents can redefine what AI makes possible. We’d love for you to join our lab and build it from the ground up! Key job responsibilities You will contribute directly to AI agent development in an applied research role, including model training, dataset design, and pre- and post-training optimization. You will be hired as a Member of Technical Staff.
US, WA, Seattle
As part of the AWS Solutions organization, we have a vision to provide business applications, leveraging Amazon’s unique experience and expertise, that are used by millions of companies worldwide to manage day-to-day operations. We will accomplish this by accelerating our customers’ businesses through delivery of intuitive and differentiated technology solutions that solve enduring business challenges. We blend vision with curiosity and Amazon’s real-world experience to build opinionated, turnkey solutions. Where customers prefer to buy over build, we become their trusted partner with solutions that are no-brainers to buy and easy to use. The Team Just Walk Out (JWO) is a new kind of store with no lines and no checkout—you just grab and go! Customers simply use the Amazon Go app to enter the store, take what they want from our selection of fresh, delicious meals and grocery essentials, and go! Our checkout-free shopping experience is made possible by our Just Walk Out Technology, which automatically detects when products are taken from or returned to the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. Shortly after, we’ll charge your account and send you a receipt. Check it out at amazon.com/go. Designed and custom-built by Amazonians, our Just Walk Out Technology uses a variety of technologies including computer vision, sensor fusion, and advanced machine learning. Innovation is part of our DNA! Our goal is to be Earths’ most customer centric company and we are just getting started. We need people who want to join an ambitious program that continues to push the state of the art in computer vision, machine learning, distributed systems and hardware design. Key job responsibilities Everyone on the team needs to be entrepreneurial, wear many hats and work in a highly collaborative environment that’s more startup than big company. We’ll need to tackle problems that span a variety of domains: computer vision, image recognition, machine learning, real-time and distributed systems. As an Applied Scientist, you will help solve a variety of technical challenges and mentor other scientists. You will tackle challenging, novel situations every day and given the size of this initiative, you’ll have the opportunity to work with multiple technical teams at Amazon in different locations. You should be comfortable with a degree of ambiguity that’s higher than most projects and relish the idea of solving problems that, frankly, haven’t been solved at scale before - anywhere. Along the way, we guarantee that you’ll learn a ton, have fun and make a positive impact on millions of people. A key focus of this role will be developing and implementing advanced visual reasoning systems that can understand complex spatial relationships and object interactions in real-time. You'll work on designing autonomous AI agents that can make intelligent decisions based on visual inputs, understand customer behavior patterns, and adapt to dynamic retail environments. This includes developing systems that can perform complex scene understanding, reason about object permanence, and predict customer intentions through visual cues. About the team AWS Solutions As part of the AWS solutions organization, we have a vision to provide business applications, leveraging Amazon's unique experience and expertise, that are used by millions of companies worldwide to manage day-to-day operations. We will accomplish this by accelerating our customers' businesses through delivery of intuitive and differentiated technology solutions that solve enduring business challenges. we blend vision with curiosity and Amazon's real-world experience to build opinionated, turnkey solutions. Where customers prefer to buy over build, we become their trusted partner with solutions that are no-brainers to buy and easy to use. About AWS Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture AWS values curiosity and connection. Our employee-led and company-sponsored affinity groups promote inclusion and empower our people to take pride in what makes us unique. Our inclusion events foster stronger, more collaborative teams. Our continual innovation is fueled by the bold ideas, fresh perspectives, and passionate voices our teams bring to everything we do. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve.
US, MA, Boston
We're a new research lab based in San Francisco and Boston focused on developing foundational capabilities for useful AI agents. We're pursuing several key research bets that will enable AI agents to perform real-world actions, learn from human feedback, self-course-correct, and infer human goals. We're particularly excited about combining large language models (LLMs) with reinforcement learning (RL) to solve reasoning and planning, learned world models, and generalizing agents to physical environments. We're a small, talent-dense team with the resources and scale of Amazon. Each team has the autonomy to move fast and the long-term commitment to pursue high-risk, high-payoff research. AI agents are the next frontier—the right research bets can reinvent what's possible. Join us and help build this lab from the ground up. Key job responsibilities * Define the product vision and roadmap for our agentic developer platform, translating research into products developers love * Partner deeply with research and engineering to identify which capabilities are ready for productization and shape how they're exposed to customers * Own the developer experience end-to-end from API design and SDK ergonomics to documentation, sample apps, and onboarding flows * Understand our customers deeply by engaging directly with developers and end-users, synthesizing feedback, and using data to drive prioritization * Shape how the world builds AI agents by defining new primitives, patterns, and best practices for agentic applications About the team Our team brings the AGI Lab's agent capabilities to customers. We build accessible, usable products: interfaces, frameworks, and solutions, that turn our platform and model capabilities into AI agents developers can use. We own the Nova Act agent playground, Nova Act IDE extension, Nova Act SDK, Nova Act AWS Console, reference architectures, sample applications, and more.
CA, ON, Toronto
The Sponsored Products and Brands (SPB) team at Amazon Ads is re-imagining the advertising landscape through state-of-the-art generative AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences, bridging human creativity with artificial intelligence to transform every aspect of the advertising lifecycle from ad creation and optimization to performance analysis and customer insights. We are a passionate group of innovators dedicated to developing responsible and intelligent AI technologies that balance the needs of advertisers, enhance the shopping experience, and strengthen the marketplace. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising. The Off-Search team within Sponsored Products and Brands (SPB) is focused on building delightful ad experiences across various surfaces beyond Search on Amazon—such as product detail pages, the homepage, and store-in-store pages—to drive monetization. Our vision is to deliver highly personalized, context-aware advertising that adapts to individual shopper preferences, scales across diverse page types, remains relevant to seasonal and event-driven moments, and integrates seamlessly with organic recommendations such as new arrivals, basket-building content, and fast-delivery options. To execute this vision, we work in close partnership with Amazon Stores stakeholders to lead the expansion and growth of advertising across Amazon-owned and -operated pages beyond Search. We operate full stack—from backend ads-retail edge services, ads retrieval, and ad auctions to shopper-facing experiences—all designed to deliver meaningful value. Curious about our advertising solutions? Discover more about Sponsored Products and Sponsored Brands to see how we’re helping businesses grow on Amazon.com and beyond! Key job responsibilities This role will be pivotal in redesigning how ads contribute to a personalized, relevant, and inspirational shopping experience, with the customer value proposition at the forefront. Key responsibilities include, but are not limited to: - Contribute to the design and development of GenAI, deep learning, multi-objective optimization and/or reinforcement learning empowered solutions to transform ad retrieval, auctions, whole-page relevance, and/or bespoke shopping experiences. - Collaborate cross-functionally with other scientists, engineers, and product managers to bring scalable, production-ready science solutions to life. - Stay abreast of industry trends in GenAI, LLMs, and related disciplines, bringing fresh and innovative concepts, ideas, and prototypes to the organization. - Contribute to the enhancement of team’s scientific and technical rigor by identifying and implementing best-in-class algorithms, methodologies, and infrastructure that enable rapid experimentation and scaling. - Mentor and grow junior scientists and engineers, cultivating a high-performing, collaborative, and intellectually curious team. A day in the life As an Applied Scientist on the Sponsored Products and Brands Off-Search team, you will contribute to the development in Generative AI (GenAI) and Large Language Models (LLMs) to revolutionize our advertising flow, backend optimization, and frontend shopping experiences. This is a rare opportunity to redefine how ads are retrieved, allocated, and/or experienced—elevating them into personalized, contextually aware, and inspiring components of the customer journey. You will have the opportunity to fundamentally transform areas such as ad retrieval, ad allocation, whole-page relevance, and differentiated recommendations through the lens of GenAI. By building novel generative models grounded in both Amazon’s rich data and the world’s collective knowledge, your work will shape how customers engage with ads, discover products, and make purchasing decisions. If you are passionate about applying frontier AI to real-world problems with massive scale and impact, this is your opportunity to define the next chapter of advertising science. About the team The Off-Search team within Sponsored Products and Brands (SPB) is focused on building delightful ad experiences across various surfaces beyond Search on Amazon—such as product detail pages, the homepage, and store-in-store pages—to drive monetization. Our vision is to deliver highly personalized, context-aware advertising that adapts to individual shopper preferences, scales across diverse page types, remains relevant to seasonal and event-driven moments, and integrates seamlessly with organic recommendations such as new arrivals, basket-building content, and fast-delivery options. To execute this vision, we work in close partnership with Amazon Stores stakeholders to lead the expansion and growth of advertising across Amazon-owned and -operated pages beyond Search. We operate full stack—from backend ads-retail edge services, ads retrieval, and ad auctions to shopper-facing experiences—all designed to deliver meaningful value. Curious about our advertising solutions? Discover more about Sponsored Products and Sponsored Brands to see how we’re helping businesses grow on Amazon.com and beyond!
US, MA, N.reading
Amazon Industrial Robotics is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at an unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic dexterous manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. At Amazon Industrial Robotics we leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at an unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. We are pioneering the development of dexterous manipulation system that: - Enables unprecedented generalization across diverse tasks - Enables contact-rich manipulation in different environments - Seamlessly integrates low-level skills and high-level behaviors - Leverage mechanical intelligence, multi-modal sensor feedback and advanced control techniques. The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. Key job responsibilities - Design and implement methods for dexterous manipulation - Design and implement methods for use of dexterous end effectors with force and tactile sensing - Develop a hierarchical system that combines low-level control with high-level planning - Utilize state-of-the-art manipulation models and optimal control techniques
US, MA, Boston
AI is the most transformational technology of our time, capable of tackling some of humanity’s most challenging problems. That is why Amazon is investing in generative AI (GenAI) and the responsible development and deployment of large language models (LLMs) across all of our businesses. Come build the future of human-technology interaction with us. We are looking for a Research Scientist with strong technical skills which includes coding and natural language processing experience in dataset construction, training and evaluating models, and automatic processing of large datasets. You will play a critical role in driving innovation and advancing the state-of-the-art in natural language processing and machine learning. You will work closely with cross-functional teams, including product managers, language engineers, and other scientists. Key job responsibilities Specifically, the Research Scientist will: • Ensure quality of speech/language/other data throughout all stages of acquisition and processing, including data sourcing/collection, ground truth generation, normalization, transformation, cross-lingual alignment/mapping, etc. • Clean, analyze and select speech/language/other data to achieve goals • Build and test models that elevate the customer experience • Collaborate with colleagues from science, engineering and business backgrounds • Present proposals and results in a clear manner backed by data and coupled with actionable conclusions • Work with engineers to develop efficient data querying infrastructure for both offline and online use cases
US, MA, Boston
AI is the most transformational technology of our time, capable of tackling some of humanity’s most challenging problems. That is why Amazon is investing in generative AI (GenAI) and the responsible development and deployment of large language models (LLMs) across all of our businesses. Come build the future of human-technology interaction with us. We are looking for a Research Scientist with strong technical skills which includes coding and natural language processing experience in dataset construction, training and evaluating models, and automatic processing of large datasets. You will play a critical role in driving innovation and advancing the state-of-the-art in natural language processing and machine learning. You will work closely with cross-functional teams, including product managers, language engineers, and other scientists. Key job responsibilities Specifically, the Research Scientist will: • Ensure quality of speech/language/other data throughout all stages of acquisition and processing, including data sourcing/collection, ground truth generation, normalization, transformation, cross-lingual alignment/mapping, etc. • Clean, analyze and select speech/language/other data to achieve goals • Build and test models that elevate the customer experience • Collaborate with colleagues from science, engineering and business backgrounds • Present proposals and results in a clear manner backed by data and coupled with actionable conclusions • Work with engineers to develop efficient data querying infrastructure for both offline and online use cases
US, WA, Bellevue
This is currently a 12 month temporary contract opportunity with the possibility to extend to 24 months based on business needs. The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
This is currently a 12 month temporary contract opportunity with the possibility to extend to 24 months based on business needs. The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Seattle
Amazon Music is an immersive audio entertainment service that deepens connections between fans, artists, and creators. From personalized music playlists to exclusive podcasts, concert livestreams to artist merch, Amazon Music is innovating at some of the most exciting intersections of music and culture. We offer experiences that serve all listeners with our different tiers of service: Prime members get access to all the music in shuffle mode, and top ad-free podcasts, included with their membership; customers can upgrade to Amazon Music Unlimited for unlimited, on-demand access to 100 million songs, including millions in HD, Ultra HD, and spatial audio; and anyone can listen for free by downloading the Amazon Music app or via Alexa-enabled devices. Join us for the opportunity to influence how Amazon Music engages fans, artists, and creators on a global scale. We are seeking a highly skilled and analytical Research Scientist. You will play an integral part in the measurement and optimization of Amazon Music marketing activities. You will have the opportunity to work with a rich marketing dataset together with the marketing managers. This role will focus on developing and implementing causal models and randomized controlled trials to assess marketing effectiveness and inform strategic decision-making. This role is suitable for candidates with strong background in causal inference, statistical analysis, and data-driven problem-solving, with the ability to translate complex data into actionable insights. As a key member of our team, you will work closely with cross-functional partners to optimize marketing strategies and drive business growth. Key job responsibilities Develop Causal Models Design, build, and validate causal models to evaluate the impact of marketing campaigns and initiatives. Leverage advanced statistical methods to identify and quantify causal relationships. Conduct Randomized Controlled Trials Design and implement randomized controlled trials (RCTs) to rigorously test the effectiveness of marketing strategies. Ensure robust experimental design and proper execution to derive credible insights. Statistical Analysis and Inference Perform complex statistical analyses to interpret data from experiments and observational studies. Use statistical software and programming languages to analyze large datasets and extract meaningful patterns. Data-Driven Decision Making Collaborate with marketing teams to provide data-driven recommendations that enhance campaign performance and ROI. Present findings and insights to stakeholders in a clear and actionable manner. Collaborative Problem Solving Work closely with cross-functional teams, including marketing, product, and engineering, to identify key business questions and develop analytical solutions. Foster a culture of data-informed decision-making across the organization. Stay Current with Industry Trends Keep abreast of the latest developments in data science, causal inference, and marketing analytics. Apply new methodologies and technologies to improve the accuracy and efficiency of marketing measurement. Documentation and Reporting Maintain comprehensive documentation of models, experiments, and analytical processes. Prepare reports and presentations that effectively communicate complex analyses to non-technical audiences.