QE Specialist - AI Factory CoE

Definity Financial • Waterloo, ON, Canada • 4d ago

As a Quality Engineering Specialist within the AI Factory Centre of Excellence (CoE), you will play a critical role in ensuring quality, reliability, and scalability of AI-driven solutions. You will contribute to the end-to-end quality engineering strategy for AI/ML models and AI-powered applications, ensuring robust test automation and quality processes that support continuous delivery.

Key Responsibilities:

Must Have:

Test Automation & Shift-left Approach: maximize test automation coverage by embedding test automation frameworks, synthetic data creation and shifting left in the delivery pipeline.
Functional UI & Microservices/API Test Automation: design and build reusable test automation assets for validating functionality through the UI and/or the API layer.
Quality & Compliance: ensure applications adhere to business and technical specifications and risk management guidelines by building a robust test design.
Agile & CI/CD Integration: embed testing strategies into Agile methodologies, leveraging CI/CD pipelines for continuous validation and automated testing.
Defect & Issue Resolution: provide real-time quality insights by tracking test status, code quality deviations, and potential business impact, driving timely resolution.
Cross-Functional Collaboration: partner with Data Scientists, Developers, DevOps, and Business Teams to integrate QE into AI development and deployment pipelines.

Nice to Have:

Data Testing: define and implement tests for validating data integrity, accuracy and completeness including data ingestion, ETL validation, data integrity testing, and cloud-based analytics testing.
AI Testing Innovation: recommend and solution modern AI testing tools, frameworks, and methodologies, fostering continuous improvements in QE practices.
AI/ML Model Validation and Testing: develop and execute tests against AI/ML models, including accuracy validation and bias detection.

What do you bring to the role?

Must Have:

Undergraduate degree in Computer Science, Data Science, Engineering or a related discipline.
7+ years’ experience in Quality Engineering with a focus on test automation framework design and scalable implementation
Knowledge of AI use cases, AI/ML models, model retraining and AI governance best practices.
Strong background in full stack testing including functional UI, API and Big Data.
Experience in Agile development, JIRA, Confluence, and test management tools (Zephyr preferred).
Experience in financial services, insurance, or other regulated industries is an asset.
Goal oriented; must be able to remove roadblocks and deliver goals.
Proven ability to adapt quickly and easily to change.
Adapt to work in Hybrid work model is must.

Nice To Have:

Hands-on test experience with AI, testing of AI/ML models, model retraining and AI governance best practices.
Experience with GCP (preferred), AWS, or Azure data platforms.
Expertise in AI-powered test automation frameworks and CI/CD integration.
Knowledge of AI model monitoring and continuous testing within DevOps pipelines.

Salary Range: $76,100 - $146,000