Director, Product Management AI Quality & Evaluation
FactSetMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Lead the development and operation of FactSet's shared AI evaluation platform, owning its roadmap, adoption strategy, and integration into product workflows.
Define and maintain standardized evaluation methodologies including quality taxonomies, golden pathways, and minimum quality thresholds across AI capabilities.
Build and manage the human review operations and federated AI evaluation practice to ensure consistent quality standards and continuous improvement across teams.
Minimum Requirements
10+ years in product management with at least 4+ years focused on AI/ML or data-intensive products and experience managing product managers.
Proven experience defining evaluation methodologies for LLM or ML systems, including metrics, failure taxonomies, and quality thresholds that were adopted by engineering teams.
Demonstrated success in launching an internal platform from inception to enterprise-wide adoption, managing migration, enablement, and change management.
Work Experience Required: 10+ years in product management including 4+ years on AI/ML or data-intensive products.
Ideal Candidate Profile
Experienced leader with a track record of building AI/ML evaluation standards and platforms inside large organizations rather than inheriting them.
Comfortable operating in a paired product-engineering leadership model with strong influence over cross-team adoption without formal authority.
Familiar with modern AI evaluation techniques (offline/online eval, human-in-the-loop, LLM-as-judge) and observability tooling, with strong judgment to maintain quality bar under commercial pressure.