Applied AI Engineer

About Pareto

Humanity is in a virtuous cycle: human insight improves AI, and better AI expands what people can do. Sustaining it depends on the one input that can't be automated: expert human judgment https://pareto.ai/blog/debating-persuasive-llms-truthful-answers.

At Pareto, we build the platform that turns that judgment into the data https://pareto.ai/blog/community-driven-knowledge-resource-ai, evals https://pareto.ai/blog/introducing-attunebench, and RL environments frontier models https://pareto.ai/blog/llm-metacognition-shared-and-shallow learn from. We work with leading frontier labs like Anthropic and GDM, and we give skilled people everywhere a way to shape the future of AI and share in what it creates.

This RL environment and human-data infrastructure is already in production. Our job now is to scale it.

Responsibilities

• Design and build the pipelines that generate synthetic tasks and evaluation environments for AI model training — this is the factory floor of AI development, producing training fuel for next-generation models, not the models themselves

• Architect the workflows where AI and humans work together in the loop — deciding what gets automated, what requires human intervention, how state is preserved across handoffs, and how the whole system stays reliable at scale

• Own and lead the most complex system design discussions — produce one-page technical scoping documents that surface hidden risks before development begins, define technology stacks, and establish engineering guidelines that let the team move fast without breaking things

• Rapidly assess whether a technical idea is worth building — get early signal, align stakeholders, and kill or accelerate accordingly

• Partner closely with research, operations, and data teams — juggle multiple workstreams, make smart tradeoff decisions as priorities shift, and translate ambiguous business needs into concrete technical architecture

• Build reusable frameworks and engineering guidelines that raise the team's collective execution muscle

• You may be a good fit if you have

• 8+ years of software engineering experience with a track record of owning complex systems end-to-end

• A software engineering foundation first — you think in systems, architecture, and engineering tradeoffs, not in models and experiments

• Production experience building and shipping agentic workflows, multi-agent orchestration, HITL pipelines, and LLM-powered applications with measurable business outcomes — RAG, vector stores, semantic search, and multi-model LLM stacks in production, not just demos

• Battle-tested context engineering practices — you reason clearly about the limits of AI and architect around them

• Experience with distributed systems architecture applied to AI or data platforms — reliable, observable, and scalable systems built in service of a product

• Daily proficiency with agentic coding tools (Claude Code, Cursor, or equivalent) — you use these to multiply your output, not pad it

• A track record of operating in ambiguity — shipping fast, pivoting when wrong, and moving on without ego

• Exceptional written and verbal English communication skills — you can lead a design discussion, push back on stakeholders, and document architecture clearly. Communication cannot be a bottleneck

• Nice to Have

• Experience at an AI data company (Scale AI, Surge, Snorkel, Labelbox, or similar) — particularly building synthetic data pipelines, eval environments, or task generation systems. This is the dream background.

• Experience building human data labeling interfaces, annotation workflows, or data collection pipelines

• Familiarity with preference data and reward models used in AI model training (RLHF, RLVR, or similar)

• Proficiency with our stack: Python, TypeScript, AWS, GCP, Terraform, Temporal Cloud, containerization, LLM gateways, RAG frameworks, and data pipeline tooling

• Ability to employ data structures and algorithms when forming AI/LLM solutions

• Ability to reason about requirements with a bias for Essentialism