Current jobs related to Bilingual LLM Evaluator - Greater Gothenburg Metropolitan Area - Mercor

  • LLM Context Engineer

    2 weeks ago

    Greater Stockholm Metropolitan Area, Sweden ContextClone Full time

    Company DescriptionContextClone Hive is the premier LLM (Large Language Model) Context Engineering Operating System, designed to address the challenges of managing AI systems that go beyond human intelligence. Focused on equipping humanity with tools to maintain control over advanced AI, ContextClone Hive ensures safe and effective interaction with these...

  • Applied AI Scientist

    4 days ago

    Greater Stockholm Metropolitan Area, Sweden Etos Full time

    Applied AI Scientist - EtosLocation:Stockholm, Sweden (On-site)Company:Etos - Educational Technology Software DevelopmentAbout EtosAt Etos, we're defining the new standard for AI in the classroom. We provide the secure infrastructure that makes AI deployment possible—and the integrity layers that make it responsible. From private LLM orchestration to...

  • Production Manager

    2 weeks ago

    Greater Stockholm Metropolitan Area, Sweden Atonemo Full time

    PLEASE NOTE: THIS IS AN ON-SITE ROLE IN SHENZHEN, CHINAAPPLICANTS MUST BE BASED IN CHINA, ALL APPLICATIONS FROM OUTSIDE CHINA WILL NOT BE REVIEWEDProduction Manager / Product Engineer (Shenzhen, China)About the RoleAtonemo is a Swedish consumer electronics company that operates within the world of audio.We want to democratize the speaker market and with our...

  • Business Development Representative

    24 hours ago

    Greater Stockholm Metropolitan Area, Sweden Medius Full time

    About UsAt Medius, we believe managing finance should be about strategy, not stress. That same mindset shapes not only the solutions we build, but also the culture we create for our people. We remove complexity, embrace innovation, and give our teams the freedom to focus on what truly matters — whether that's transforming the future of finance with AI or...

  • Senior AI/Machine Learning Developer

    2 weeks ago

    Gothenburg, Västra Götaland, Sweden Cyber Instincts AB Full time

    Cyber Instincts AB is a dynamic, resourceful and innovative IT and business consulting firm. We are an exclusive consulting firm with strong customer focus and a vibrant employee-centric culture. Customer success is an important goal for us, and we find that our biggest strength in achieving that goal is in our people. Hence, we genuinely strive to ensure...

  • Thesis worker

    2 weeks ago

    Gothenburg, Västra Götaland, Sweden e0-eb3a-49ce-af66-e0eb4e594d15 Full time

    Transport is at the core of modern society. Imagine using your expertise to shape sustainable transport and infrastructure solutions for the future. If you seek to make a difference on a global scale, working with next-gen technologies and the sharpest collaborative teams, then we could be a perfect match.BackgroundThe Volvo Performance System is built on...

  • Thesis work

    2 weeks ago

    Gothenburg, Västra Götaland, Sweden Volvo Cars Full time

    Thesis Worker at Volvo CarsWelcome to explore the world of Volvo Cars by writing your thesis with us As a thesis worker in our organization you are supported by a supervisor who follows you during your project. Through your thesis work you will be able to contribute to our company purpose – providing freedom to move in a safe, sustainable and personal way...

  • Senior Machine Learning Research Engineer

    2 weeks ago

    Gothenburg, Västra Götaland, Sweden AI Sweden Full time

    AI Sweden is the national center for applied artificial intelligence, funded jointly by the public and private sectors. Our mission is to accelerate the use of AI to strengthen society, enhance competitiveness, and benefit everyone living in Sweden. A key part of this mission is tackling major research and innovation challenges together with our partners.We...

  • Senior Software Engineer

    1 week ago

    Gothenburg, Västra Götaland, Sweden Recorded Future Full time

    With 1,000+ intelligence professionals serving over 1,900 clients worldwide, Recorded Future is the world's most advanced, and largest, intelligence company We're looking for an exceptional backend developer to join Recorded Future's Backend Services team. As a Senior Software Engineer, you'll work alongside a talented group of engineers who are passionate...

  • Computer Vision Deep Learning Engineer IRC283302

    2 weeks ago

    Gothenburg, Västra Götaland, Sweden GlobalLogic Full time

    DescriptionThe client's purpose is to make safe and intelligent mobility real, for everyone, everywhere. The company develops the complete software stack for advanced driver assistance systems and autonomous driving, from sensing to actuation. Their focus is to build a single, cutting-edge software platform in order to serve various levels of autonomy and...

Bilingual LLM Evaluator

2 weeks ago

Greater Gothenburg Metropolitan Area, Sweden Mercor Full time

About The Job
Mercor
connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include
Benchmark
,
General Catalyst
,
Peter Thiel
,
Adam D'Angelo
,
Larry Summers
, and
Jack Dorsey
.

Position:
AI Model Evaluator

Type:
Full-time or Part-time Contract Work
Compensation:
$40/hour
Location:
Geography restricted to Europe, USA
Role Responsibilities

  • Evaluate LLM-generated responses on their ability to effectively answer user queries.
  • Conduct fact-checking using trusted public sources and external tools.
  • Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies.
  • Assess reasoning quality, clarity, tone, and completeness of responses.
  • Ensure model responses align with expected conversational behavior and system guidelines.
  • Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines.

Qualifications
Must-Have

  • Bachelor's degree
  • Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in German
  • Significant experience using large language models (LLMs)
  • Excellent writing skills
  • Strong attention to detail
  • Adaptable and comfortable moving across topics, domains, and customer requirements
  • Background or experience in domains requiring structured analytical thinking
  • Excellent college-level mathematics skills

Preferred

  • Prior experience with RLHF, model evaluation, or data annotation work
  • Experience writing or editing high-quality written content
  • Experience comparing multiple outputs and making fine-grained qualitative judgments
  • Familiarity with evaluation rubrics, benchmarks, or quality scoring systems

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
,