Location
toronto
Job Type
Full-time
Posted
June 01, 2026
Job Description
About the Role & Team IDC is building the next generation of AI-powered intelligence platforms that transform how technology decisions get made. Our platform re-imagines the way decision-makers discover and interact with trusted research and data, and is foundational to IDC's future.
What You’ll Do
Design and build the evaluation infrastructure that ensures the platform's AI systems produce accurate, well-sourced, high-quality responses
Build automated test suites that validate answer quality across agent pipeline changes
Develop regression detection systems that catch quality degradation before it reaches users
Create evaluation frameworks that measure response accuracy, citation correctness, and source quality
Work closely with the product team to translate quality criteria into measurable, automatable test scenarios
Build cost and latency monitoring that tracks the operational efficiency of AI pipeline execution
What You’ll Do
Design and build the evaluation infrastructure that ensures the platform's AI systems produce accurate, well-sourced, high-quality responses
Build automated test suites that validate answer quality across agent pipeline changes
Develop regression detection systems that catch quality degradation before it reaches users
Create evaluation frameworks that measure response accuracy, citation correctness, and source quality
Work closely with the product team to translate quality criteria into measurable, automatable test scenarios
Build cost and latency monitoring that tracks the operational efficiency of AI pipeline execution
Ready to Apply?
Submit your application for AI Engineer – AI Observability and Quality at IDC
Apply Now