attention head AI jobs Alerts Saved
Jobs › XM

Senior AI QA Test Automation Engineer – CX Agent

XM · Limassol; Nicosia; Athens; Thessaloniki; Fully Remote - in other eligible locations
Apply on XM's site
Tailor my resume
Posted
Sep 11, 2026
Last verified
4 h ago
Employment
Full-time
Level
Senior
Function
AI/ML Engineering
AI roleRemotePythonGoTypeScriptKubernetesTerraformAWSGCPAzureLLMsRAGEvals

The Role:
We're seeking a Senior AI QA Test Automation Engineer to take a leading technical role in AI quality and test automation. You will work primarily with our AWS Bedrock AgentCore-based CX Agent Suite, a multi-agent customer-support system covering intent routing, FAQs, deposit-status queries, and human handoff. You will design and build the evaluation and QA tooling required to validate AI agents reliably, while also contributing to technical standards and best practices across QA.
This is a Senior Individual Contributor (IC) role with a strong technical and strategic focus. You'll work closely with QA leadership, Data Science, and Engineering to architect scalable AI evaluation systems, establish technical standards, and drive the evolution of our AI evaluation framework. Our current evaluation harness is based on DeepEval and is evolving towards a broader agentic evaluation architecture. You will have the opportunity to evaluate and prototype emerging agent-orchestration approaches, including Strands Agents, and contribute to architectural decisions around their adoption.

The main responsibilities of the position include:

Main requirements:

The following will be considered an advantage:

Benefit from:

The Hiring Experience: What Awaits You

All applications will be treated with strict confidentiality!

By submitting this application, I confirm that I have read and agree to the terms of the Privacy and Data Protection Notice and the Confidentiality Undertaking.

Apply on XM's site

Attention Head lists this role as published on XM's own job board. Applications go to the employer.

Not this one? Get roles like it weekly

A job alert: the week's best new AI roles from employers' own job boards, every Monday. Unsubscribe in one click.

Similar roles at other employers

All senior AI/ML Engineering roles →

More roles at XM

All roles at XM →