data
Posted Oct 22, 2025Senior Research Scientist, Model Evaluation
at Cohere
Toronto, CanadaRemote
Responsibilities
- you will: - Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish. - Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations. - Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency. - Build scalable and reusable tools for digging into model performance.
Requirements
- We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents.
- We believe that our work is instrumental to the widespread adoption of AI.
- You may be a good fit if: - You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities. - You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality. - You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about. - You have strong software engineering skills.
Benefits
- Full-Time Employees at Cohere enjoy these Perks: 🤝 An open and inclusive culture and work environment 🧑💻 Work closely with a team on the cutting edge of AI research 🍽 Weekly lunch stipend, in-office lunches & snacks 🦷 Full health and dental benefits, including a separate budget to take care of your mental health 🐣 100% Parental Leave top-up for up to 6 months 🎨 Personal enrichment
- benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement 🏙 Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend ✈️ 6 weeks of vacation (30 working days!)
Contact
- Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form https://docs.google.com/forms/d/12a6IrLdF3kI2nonKSr4tiFuz18rLQbaeYV-JM9L4o9Q/edit, and we will work together to meet your needs.