Lead Quality Engineer - R01573517
Brillio · Bangalore, Karnataka, India
Apply on Brillio's siteTailor my resume
Posted
Nov 13, 2026Last verified
5 h agoEmployment
EmployeeLevel
SeniorFunction
Software EngineeringAI rolePythonGCPSQLEvals
Lead Quality Engineer
Primary Skills
- Test Data Management, Jenkins, Regression Testing, Postman, Selenium, Functional Testing, BDD - Behavior Driven Development, SoapUI, System Integration testing, Application Testing, Pixel Testing, Cross Browser Testing, Zephyr, TDD - Test Driven Development, Accessibility Testing (JAWS), Visual Testing, Cucumber
Specialization
- Quality Engineering - Product Applications & Microservices: Lead Quality Engineer
Job requirements
- Experience Range: with at least 4 to 6 years of experience in quality engineering, including hands-on testing and evaluation of AI-powered product applications and microservices Key Responsibilities: 1. Develop and execute robust evaluation frameworks for AI agents, focusing on Responsible AI and safety metrics such as bias, toxicity, and alignment failures 2. Design synthetic testing environments and simulation strategies to stress-test agents under diverse real-world conditions 3. Implement adversarial testing, red-teaming, and prompt injection methodologies to identify vulnerabilities and ensure agent robustness 4. Integrate evaluation pipelines into CI/CD workflows for continuous quality assurance and rapid iteration 5. Collaborate with policy and legal teams to ensure agent evaluations meet regulatory compliance and ethical standards 6. Analyze evaluation results and provide actionable insights to development teams, prioritizing safety and risk mitigation 7. Configure and execute automated test suites using Python and relevant AI/ML testing libraries (e.g., PyTest, DeepEval, Ragas, Giskard) 8. Manage bug detection, documentation, and tracking using Jira, focusing on performance degradations and alignment failures Required Skills: 1. Python programming 2. PyTest 3. DeepEval 4. Ragas 5. Giskard 6. Langsmith 7. Google Agent Development Kit (ADK) 8. Vertex AI 9. Safety evaluations (red teaming, adversarial testing) 10. Prompt injection testing 11. API testing for AI/ML systems 12. Data analysis with Pandas and SQL Preferred Skills: 1. Gemini models 2. Hugging Face 3. Garbage-in/Garbage-out (GIGO) testing tools 4. Data poisoning techniques 5. AI ethics principles and policy background 6. Visualization tools for evaluation results Desired Qualifications: 1. Bachelor's degree in Computer Science, Artificial Intelligence, Information Technology, or Software Engineering 2. ISTQB Foundation Level certification or Certified Software Tester (CST) 3. Certification in AI/ML testing or Responsible AI (e.g., Google Cloud Certified - Professional Machine Learning Engineer)
Attention Head lists this role as published on Brillio's own job board. Applications go to the employer.
Similar roles at other employers
- Senior Python Full Stack Engineer (6 Yrs - Python, Azure/AWS, LLM)Libra Technology GmbH · IND - Chennai, Neville TowersNov 20, 2026
- Senior Engineer - DevOps & Platform Engineering (AI-Driven Developer Productivity) - BengaluruOmnissa International Unlimited Company · Bengaluru, IndiaNov 20, 2026
- Application Engineer - III /Sr. AI Application EngineerThe Vanguard Group, Inc. · Hyderabad, IndiaNov 20, 2026
- Senior Software Engineer, Salesforce & AIF5 Networks Australia Pty. Limited · Hyderabad (SEZ)Nov 19, 2026
- Engineer Lead, Artificial Intelligence / Machine Learning- 8 to 12 Years- BangaloreFIS INC (OLD FIS) · IND BNGL FL2-3 TWR 3Nov 19, 2026
All senior Software Engineering roles in India →
More roles at Brillio
- Lead AI Engineer I - R01573927Bangalore, Karnataka, IndiaNov 20, 2026
- Data Specialist - R01573995San Francisco, California, United StatesNov 17, 2026
- Data Architect - R01573991San Francisco, California, United StatesNov 17, 2026
- Databricks Architect - R01573990San Francisco, California, United StatesNov 17, 2026
- Databricks Data Architect - R01573925Bangalore, Karnataka, IndiaNov 17, 2026
- Snowflake Data Architect - R01573923Bangalore, Karnataka, India; Pune, Maharashtra, IndiaNov 17, 2026
- AWS Data Architect - R01573922Bangalore, Karnataka, IndiaNov 17, 2026
- Senior Lead Data Engineer - R01573561Bangalore, Karnataka, IndiaNov 13, 2026