attention head AI jobs Alerts Saved
Jobs › Ellipsis Health

Senior Backend Engineer, AI Evaluations

Ellipsis Health · San Francisco - Hybrid
Apply on Ellipsis Health's site
Tailor my resume
Posted
May 13, 2026
Last verified
4 h ago
Salary
$175K – $250K / year
Employment
FullTime
Level
Senior
Function
Software Engineering
AI roleRemotePythonTypeScriptLLMsEvals

6% above the median for senior Software Engineering roles in the United States · median $200K a year, from 1,264 roles that state pay

Open 6 months. Roles open this long are often filled, or kept open all year. Check the date on the employer's site before you apply.

Not mentioned in the posting. Ellipsis Health filed 5 H-1B, H-1B1 or E-3 applications for AI roles in FY2025–FY2026, one for a worker changing employer. Its US visa filings

ABOUT THE ROLE

We’re seeking a strong Senior Backend Engineer to join our AI Evaluation team, focused on building the infrastructure and internal tooling that enable reliable, repeatable evaluation of AI systems in production. In this role, you’ll develop the scaffolding around our evaluations platform - API/data integrations, test configuration management, and reporting tools that make evaluations easy to run, extend, and operate at scale. You’ll also build and maintain evaluation frameworks for individual system components such as ASR, LLMs, TTS, knowledge bases, and guardrails, productionalizing these frameworks to support regular analysis, regression detection, and continuous monitoring.

In addition, you’ll create debugging tools that make it easy to inspect end-to-end calls, trace failures, and surface all relevant signals in one place - empowering internal teams to diagnose issues quickly and confidently. The ideal candidate is a pragmatic, infrastructure-minded engineer who enjoys turning ad hoc analysis into durable systems, cares deeply about developer experience, holds a high bar for software-quality and takes pride in building tooling that makes complex AI systems observable, testable, and easier to operate.

RESPONSIBILITIES:

QUALIFICATIONS:

PREFERRED:

SALARY AND BENEFITS

We offer competitive salary and benefits, including 401(k) matching, health, vision, and dental insurance, and very flexible paid time off.

The typical salary range for this role is $175,000 to $250,000 USD, depending on skills, qualifications, and relevant experience.

BACKGROUND CHECKS

As a health technology company, we reserve the right to run background checks on candidates to whom we extend offers, in compliance with applicable laws. We evaluate candidates holistically and comply with all “ban the box” regulations.

ASSISTANCE

If you have a disability or require accommodations during the application or recruitment process, please contact [email protected].

Apply on Ellipsis Health's site

Attention Head lists this role as published on Ellipsis Health's own job board. Applications go to the employer.

Not this one? Get roles like it weekly

A job alert: the week's best new AI roles from employers' own job boards, every Monday. Unsubscribe in one click.

Similar roles at other employers

All senior Software Engineering roles in the United States →

More roles at Ellipsis Health

All roles at Ellipsis Health →