ai evaluation engineer

Legora seeks a tech-savvy lawyer to evaluate, stress-test, and improve AI systems used in legal work. You will review outputs for accuracy and legal reasoning, and help shape evaluation benchmarks.Collaboration with senior legal engineers, product managers, and engineers is core to the role, with emphasis on issue-spotting and quality control. The position...

Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models, working in Swedish and English. You will assess AI-generated music across genres and rate it against detailed quality standards, focusing on musicality, creativity, vocal quality, and production/mix judgments.Ideal candidates have 2+ years in music...

Mercor is hiring experienced music producers and audio engineers to evaluate generative music AI models in partnership with a leading AI lab. You will assess AI-generated music across genres and rate it against detailed quality standards, working in Swedish and English.Key responsibilities include comparing songs, labeling by genre and structure, and...

About the Role Mercor is partnering with a leading AI research lab to support a Frontier Code Agents project. Contributors help evaluate and improve frontier AI coding models through structured technical assessments. The work focuses on realistic infrastructure engineering workflows and model evaluation. Spots are limited and filling quickly on a first...

Mercor seeks experienced AI Safety Practitioners to evaluate frontier models for safety, alignment, and quality across grey-area topics. You will assess responses, apply safety policies, and help improve model behavior through structured evaluations and feedback.Responsibilities include evaluating outputs for safety across sensitive domains, refining...

Swedish AI Product Evaluator🎯 The ProjectProductive Playhouse is building a talent pool of Swedish speakers for an upcoming project testing and evaluating leading AI chatbots. The objective is to enhance response quality through direct user interaction with the various AI models. Open to freelancers based outside the U.S.   This is more than one job....

Mindrift is seeking experienced software developers to build and evaluate AI coding tasks for leading tech companies. You will create realistic environments, craft tasks from intermediate states, and write comprehensive tests to validate AI agent solutions.This project-based role emphasizes quality and fairness, with flexible timing and an emphasis on...

Redpine in Stockholm is looking for a candidate skilled in designing and operating retrieval systems that serve licensed data to AI models. This role involves building evaluation infrastructures to ensure quality, translating research into practical applications, and defining metrics that prove system effectiveness. The ideal candidate has production...

We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier AI models across complex, policy-sensitive, and ambiguous ("grey-area") topics. You will assess AI-generated responses, apply safety policies, and help improve model behavior through structured evaluations and feedback. Responsibilities Evaluate...

Mercor is hiring experienced musicians to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across a wide range of genres and rate it against detailed quality standards, working in Swedish and English.Start date: immediate; Duration: up to 6 months; Most experts work around 20 hours per week;...

We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier AI models across complex, policy-sensitive, and ambiguous ("grey-area") topics. You will assess AI-generated responses, apply safety policies, and help improve model behavior through structured evaluations and feedback.ResponsibilitiesEvaluate...

Mercor is hiring experienced musicians to evaluate generative music AI models, in partnership with a leading AI lab.You will assess AI-generated music across a wide range of genres and rate it against detailed quality standards, working in Swedish and English.Key ResponsibilitiesCompare AI-generated lyrics with published songs to identify similaritiesRate...

Mindrift, powered by Toloka, is launching a Management Consulting domain that translates real-world engagements into structured learning environments for advanced AI systems. You will join a growing team of strategy consultants from top firms to shape how AI learns high‑level business reasoning.This remote, project-based, individual-contributor role...

Polish Workers is seeking a remote AI Trainer to enhance AI-generated responses in Polish. The job involves evaluating content quality, comparing AI outputs, and ensuring relevance and accuracy.This role requires excellent Polish language skills, good English understanding, and strong analytical abilities for task evaluation. Salary is approximately 35 EUR...

Mercor invites you to craft original multiple-choice questions on Swedish law for an AI evaluation dataset. As an independent contractor, you will draft exam-style questions from your professional expertise, each with ten answer options and a worked solution. Remote, flexible hours.Requirements: law degree or Swedish equivalent, ideally 2+ years practising,...

Mercor is hiring experienced musicians to evaluate generative music AI models in partnership with a leading AI lab. You will assess AI-generated music across genres and rate it against detailed quality standards, working in Swedish and English.Key responsibilities include comparing AI-generated lyrics with published songs, rating lyric quality, creativity,...

Mercor is seeking Engineering / Platform professionals with a technical background in software engineering, infrastructure, or platform development. You will review, assess, and provide structured feedback on domain-specific documents, ensuring quality, accuracy, and relevance.The role requires at least a bachelor’s degree, 2+ years of experience, and...

Prolific is looking for an AI Trainer fluent in Lithuanian to assist in evaluating AI-generated speech. You will compare text and voice snippets, listening to audio and rating its naturalness while ensuring quality control.The ideal candidate has a strong command of Lithuanian and English, with a high attention to detail. This role is perfect for those with...

Prolific is looking for an AI Trainer fluent in Lithuanian to assist in evaluating AI-generated speech. You will compare text and voice snippets, listening to audio and rating its naturalness while ensuring quality control.The ideal candidate has a strong command of Lithuanian and English, with a high attention to detail. This role is perfect for those with...

Anyone AI is seeking skilled Full-Stack Developers for a project with a leading AI lab. This remote, part-time consultant role focuses on designing and evaluating realistic coding tasks, bug fixes, and feature implementations.Contributors will create multi-file coding tasks, write specifications, extend tests, and review peer submissions. You control your...