Senior AI Quality Engineer (LLM Evaluation & Automation) 1754

Owns the eval harness and quality gate from the beginning. This role replaces the old late-stage “Evals Specialist” model with a standing owner for measurable agent quality.

Key Responsibilities

• Build and maintain the MVP eval harness: golden tasks, exception tasks, scorecard metrics, and regression packs.

• Wire evals into CI so quality regressions fail builds and releases.

• Define and maintain release-gate thresholds with Product and the Tech Lead.

• Lay the path for later adversarial and drift-testing expansion without overbuilding MVP scope.

Must-Have Qualifications

• Experience evaluating ML, LLM, or non-deterministic syst...