Source description
About the role
You have experience in multilingual quality evaluation, localization, data curation, annotation, AI evaluation, or related fields, including text-to-text and text-to-speech experiences.
You understand language quality, cultural relevance, content quality, and user experience across multiple languages and markets.
You have experience designing or conducting structured evaluations using quality rubrics, audits, annotation projects, or review methodologies.
You are comfortable using qualitative and quantitative data to identify trends, measure quality, and make recommendations.
You are familiar with large language models (LLMs), generative AI evaluation, human-in-the-loop workflows, or LLM-as-a-judge methodologies.
You enjoy working through ambiguity and turning complex quality challenges into practical evaluation strategies.
You communicate effectively and thrive in highly cross-functional environments, collaborating with technical and non-technical partners alike.
Experience with recommendation systems, personalization, search, ranking, machine translation, generative AI, dataset creation, annotation operations, evaluator calibration, prompt testing, model evaluation, SQL, Python, dashboards, or annotation platforms is a plus.
More at Spotify
Related open roles
Senior Partner Engineer - Hardware Partnerships
SE · Hybrid
Software Engineer - Financial Engineering
Remote · Toronto
Senior Engineering Manager - ML
Remote · New York · Boston · Miami
Research Scientist - Generative Audio
Remote · New York
Research Scientist - Generative Audio
Remote · United Kingdom
Senior Engineering Manager - ML
Remote · London · Stockholm
