Model evaluation AI jobs
Model evaluation is how a team knows where a model stands. Training a new model or changing a product's prompt is only worth it if someone can show the result is better, and public benchmarks stop telling models apart once most of them score well. So much of the work is designing new tests: hard questions with checkable answers, realistic tasks for agents, and measures of behaviour that has no single right answer.
Research scientists design evaluations of capabilities and risks and study what scores really mean, including whether test questions leaked into training data. Research engineers and ML engineers build the harnesses that run evaluations throughout training and the tools that track the results. Applied AI engineers build test sets from real use and grade outputs with rubrics, people or other models. Expect statistics, care with data, and the judgement to tell when a number is misleading.
We count roles whose postings mention evals, model evaluation, or evaluation frameworks, harnesses, suites or pipelines.
1,239 open AI roles at 714 employers, from their own job boards. Showing the newest 100.
Follow Model evaluation97 new this week
- Manager, Applied AI ArchitectAnthropic · Seoul, South Korea29 min ago
- Senior AI Engineer - Enterprise Agentic SystemsRubrik Job Board · Bangalore · 5 locations4 h ago
- AI Product ManagerOpusClip · Mountain Viewup to $280K · 8 h ago
- Staff Applied AI EngineerBILL · United Statesup to $250K · 12 h ago
- Senior Software Engineer -AIUiPath · Bellevueup to $230K · 15 h ago
- Staff Research Engineer/Scientist - Conversational AIServiceNow · Santa Clara, California, United Statesup to $354K · 17 h ago
- Senior Machine Learning Engineer / Applied ScientistMicrosoft · Vancouver, BC, CAup to CA$204K · 19 h ago
- Tech Product Manager - Agentic PlatformMirakl - Labs · Bordeaux, France · 2 locations19 h ago
- Model Behavior EngineerOvertone · New York Cityup to $190K · 21 h ago
- LLM Engineer (100% Remote within Poland)Docplanner · Warsaw22 h ago
- LLM Engineer (100% Remote within Spain)Docplanner · Barcelona22 h ago
- AI EngineerNielsenIQ · Pune, MH, India1 days ago
- AI Engineer, DevEx & AutomationMirakl - Labs · Bordeaux, France · 2 locations1 days ago
- LLM Engineer — Post-Training (Remote - UK)EverAI · United Kingdom; Portugalup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - Germany)EverAI · Germany; Portugalup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - Portugal)EverAI · Spain; Portugalup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - Poland)EverAI · Poland; Portugalup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - Netherlands)EverAI · Netherlands; Portugalup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - Italy)EverAI · Italy; Portugalup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - Spain)EverAI · Spainup to €125K · 1 days ago
- LLM Engineer — Post-Training (Remote - France)EverAI · France; Portugalup to €125K · 1 days ago
- Staff AI Platform EngineerServiceNow · Hyderabad, India1 days ago
- AI Research InternOpusClip · Mountain View; Remoteup to $28/h · 1 days ago
- Applied AI Architect, Beneficial DeploymentAnthropic · Sydney, Australia1 days ago
- AI EngineerCentury Health · New York Cityup to $200K · 1 days ago
- Product Manager, AI Trip PlanningAirbnb · San Francisco, CAup to $282K · 1 days ago
- AI/ML Engineer Intern – Generative AI, PhD – Summer 2027 (Mountain View, CA)LinkedIn · Mountain View, CA, United Statesup to $75/h · 1 days ago
- Senior Officer, AI Safety & Guardrails, U.S. ProgramGates Foundation · 2 Locationsup to $295K · 1 days ago
- AI ComposerWORKSHARE LTD · Denver, CO · 2 locationsup to $150K · 1 days ago
- Senior AI EngineerDoubleVerify Solutions India Private Limited · NYC Global HQ · 2 locations1 days ago
- Principal Engineer, LLMUpstart · United States | Remoteup to $330K · 1 days ago
- Staff Software Engineer- Agentic AI - FederalServiceNow · Santa Clara, CALIFORNIA, United Statesup to $308K · 1 days ago
- Principal Conversation & Agentic Experience ArchitectTwin Health · Remote, USAup to $240K · 1 days ago
- Senior Manager, AI Engineering - A&DIFS · Itasca, Illinois, United States1 days ago
- Senior Staff Machine Learning Engineer, Ads RankingReddit · Remote - United Statesup to $372K · 1 days ago
- Senior Applied AI ResearcherGuidepoint · Toronto, Ontario, Canadaup to CA$200K · 1 days ago
- Senior AI Software Engineer, AI EnablementExtend · Remote, USup to $170K · 1 days ago
- Sr. Inference Engineer (LLM Serving)GoodData.AI · Prague1 days ago
- AI Engineer Senior, BrazilCI&T · Brazil2 days ago
- Staff ML Platform Engineer - Fintech Engineering (m/f/n)InPost · Warszawa / Kraków, Poland2 days ago
- Software Engineer, AI TransformationGoodLeap · Austin, TX; San Francisco, CA; Roseville, CA - HQ; Remote, US; New…up to $145K · 2 days ago
- Staff AI Acceleration EngineerDave · United Statesup to $289K · 2 days ago
- Senior Machine Learning Solutions ArchitectEmpower Global Business Services India Private Limited · Nationwide Remoteup to $200K · 2 days ago
- Senior/Principal AI Engineer, Agentic Systems, AI for Drug DiscoveryGenentech, Inc. · South San Franciscoup to $312K · 2 days ago
- Senior Software Engineer - Internal AI Accelerator SquadXero · AU: Sydney (45 Clarence St) · 2 locationsup to CA$210K · 2 days ago
- Applied AI Engineering LeadPeregrine Technologies · San Francisco, CAup to $320K · 2 days ago
- Principal Engineering Manager, AI CarePair Team · Remote (United States)up to $250K · 2 days ago
- Research Engineer — Post-Training, Mountain View, CAGranica · Bay Area Officeup to $240K · 2 days ago
- Applied AI Engineer, CommercialAnthropic · San Francisco, CAup to $320K · 2 days ago
- AI Engineer Mid-level, BrazilCI&T · Brazil2 days ago
- Sr. AI Product ManagerRobots and Pencils · Remote, Colombia2 days ago
- Sr. AI Engineer, Backend (Altium)Renesas Electronics · Bangalore, India2 days ago
- [Paris] Applied AI Architect, IndustriesAnthropic · Paris, Franceup to €220K · 3 days ago
- Fullstack AI Engineer (Senior) - IFS LogisticsIFS · London, England, United Kingdom3 days ago
- Backend Engineer AIAtlys · Delhi HQ3 days ago
- Senior Product Manager - Agentic FoundationCambio AI Inc. · Waterlooup to $190K · 3 days ago
- Engineering Manager - AI Agents (Agent Platform, LLMs, Evals)Oscilar · Remoteup to $305K · 3 days ago
- LLM Engineer (Data Generation)42dot · Pangyo (Software Dream Center), South Korea3 days ago
- Forward Deployed Engineering Partner Program LeadCursor (Anysphere) · San Francisco, CA; New York, NYup to $336K · 3 days ago
- Forward Deployed Engineer, PartnershipsAssemblyAI · San Franciscoup to $210K · 3 days ago
- AI Agent DeveloperNovanta Corporation · Barcelona, Spain - Keonn3 days ago
- Solutions Architect IV - AIAstreya Consultancy India Private Ltd · Remote, CAup to $180K · 3 days ago
- Senior Applied AI EngineerHandshake · San Francisco, CAup to $260K · 3 days ago
- Senior Machine Learning Engineer, Driver Refinement & ValidationWaymo · Mountain View, CA, USA; San Francisco, CA, USAup to $263K · 3 days ago
- Senior Software Engineer II - Agentic IntelligenceHoneycomb.io · Remote - Canada · 2 locations3 days ago
- ML Engineer, MarketplaceLyft · Toronto, Canadaup to CA$149K · 3 days ago
- Sr AI Software EngineerThe Trade Desk · Bellevue; New York; San Francisco; San Jose; Seattleup to $229K · 3 days ago
- Sr AI Enablement EngineerThe Trade Desk · Bellevue; New York; San Francisco; San Jose; Seattleup to $229K · 3 days ago
- Forward Deployed Engineer, Customer DeploymentsTitan · New York City; San Franciscoup to $250K · 3 days ago
- Research Lead, EvaluationsBlack Forest Labs · Freiburg (Germany)3 days ago
- Applied AI/ML Research Engineer (Generative Models)CuspAI · EMEA | Europe3 days ago
- Staff AI Engineer (Security/Infrastructure)Multiverse · London3 days ago
- Member of Technical Staff, EvaluationsXDOF · San Mateo Hybridup to $300K · 3 days ago
- Fullstack AI Engineer (Staff) - IFS LogisticsIFS · Staines-upon-Thames, England, United Kingdom4 days ago
- Senior ML Engineer - Fintech Engineering (m/f/n)InPost · Warszawa / Kraków, Poland4 days ago
- Senior ML EngineerPoland and Middle-Eastern Europe · Bulgaria; Poland; Romania4 days ago
- Lead AI EngineerALO · Beverly Hills LAND, Beverly Hills, CA (HQ-L)up to $220K · 4 days ago
- Lead Engineer - Internal AI Accelerator SquadXero · AU: Sydney (45 Clarence St) · 2 locationsup to $250K · 4 days ago
- Sr Software Engineer - AI & AnalyticsSouthwest Airlines Co. · India Office4 days ago
- Product Manager, Applied AIComputer Services, Inc. · Remote, USA4 days ago
- Senior AI Research Engineer, Agentic Systems IntelligenceNVIDIA · 6 Locations4 days ago
- Java GenAI Developer – Spring Boot, Microservices, RAG & LLMsSynechron Technologies Pvt. Ltd._INDIA Company · Bengaluru - Bellandur (GTP)4 days ago
- AI Forward Deployment EngineerSphera Solutions GmbH · IN Remote · 2 locations4 days ago
- Associate Director, AI Security ArchitectINFORMATION HANDLING SERVICES MEXICO SA DE CV · 3 Locations4 days ago
- Applied AI Architect, Beneficial Deployments (Non-Profits)Anthropic · San Francisco, CA | New York City, NYup to $315K · 4 days ago
- Sr. Machine Learning Engineer, AnonymizationMercor · New York City · 2 locationsup to $500K · 4 days ago
- Senior AI EngineerCodat · London · 2 locationsup to £120K · 4 days ago
- Senior Product Manager, AI ProductsCraft Machine Inc · US (Remote)4 days ago
- Software Engineer, AI Voice AgentAircall.io, Inc. · Madrid Office4 days ago
- AI Platform EngineerCirrus Logic · Austin, Texas4 days ago
- Member of Technical Staff - Pre-Training Evals Infra (Inference)Reflection AI · Global Office4 days ago
- Senior Software Engineer, Agentic EngineeringHeadway · Remoteup to $265K · 4 days ago
- AI Native Creative Director - EdTech (French fluent 🇫🇷)Paradox · Remote Dubai; Dublin Remote; Palma Remote; Remote Latvia; Sofia…4 days ago
- AI Native Creative Strategist - EdTech (French fluent 🇫🇷)Paradox · Remote Dubai; Dublin Remote; Palma Remote; Remote Latvia; Sofia…4 days ago
- AI Native Head Of Growth - EdTech (French fluent 🇫🇷)Paradox · Office Dubai; Office Lisbon; Palma Remote; Remote Dubai; Remote…4 days ago
- Request for Proposal: Agentic AI System Partner Discovery, Design, Build and Handover for WAZZIIGirl Effect · Kenya4 days ago
- Director AI Native Product Builder | New VenturesWellhub · Brazil (Remote)5 days ago
- Senior AI EngineerVersion 1 · Bengaluru, KA, India5 days ago
- Senior AI SpecialistCoursera · India5 days ago
- Staff ML Research Engineer, Multimodal Structure & MarengoTwelveLabs · Seoul, South Korea5 days ago