other
Posted May 25Data Annotation Specialist, Safety
at Cohere
CanadaRemote
Responsibilities
- - Create prompts and safety test cases: Write realistic prompts, user scenarios, and adversarial examples that help evaluate model behavior across safety categories and uncover unsafe, evasive, over-refusing, or policy-inconsistent responses.
- - Support quality and calibration: Identify annotation inconsistencies or unclear guidelines, and provide actionable feedback on recurring edge cases, model failures, and opportunities to improve data quality.
Requirements
- We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents.
- We believe that our work is instrumental to the widespread adoption of AI.
- Cohere is looking for Data Annotation Specialists with backgrounds and skills in Trust & Safety, Content Moderation, AI model evaluation, or related fields.
- experience applying nuanced guidelines to complex or ambiguous content.
- Successful candidates will be highly detail-oriented, comfortable evaluating safety risks across different user intents and scenarios, and able to make consistent, well-reasoned decisions with a high degree of independence after onboarding.
- experience in Content Moderation,Trust and Safety, AI data annotation, LLM evaluation, or a related analytical role, with exposure to quality assurance, red teaming, and/or prompt engineering preferred. -
- Experience applying detailed guidelines to complex and sensitive content, with strong contextual and sociocultural judgment and the ability to recognize and manage personal bias.
- - Excellent command of written English and the ability to clearly justify content evaluations, including why an output is safe, unsafe, high-quality or low-quality.
- Virtual Annotation Test - This assignment will test your written skill through various language-based tasks, such as a a writing sample, interacting with a chat bot, and more.
Experience
- You may be a good fit if you have: - 1+ years of
Benefits
- We seek candidates who are able to commit to 16 hours per week minimum at a 45 CAD/hour or 45 USD/hour contract rate, depending on your location, consisting of 30/hour base pay plus 15/hour hazard pay.
- Bonus points if you are fluent in another language! - Strong attention to detail and commitment to accuracy, with the ability to maintain consistency across high-volume and monotonous tasks. - Strong execution in a remote environment, including good time management, comfort using new tools, and the ability to work independently in a global, asynchronous team.
- Independent contractors are not eligible for health benefits or other benefits provided to employees.
- Compensation for services is provided to contractors by contractors invoicing for services provided pursuant to the terms of our agreement with the contractor.
- As an independent contractor, you should anticipate fluctuations in workflow and therefore compensation for services when Cohere does not require as many hours of services in a week.
- Full-Time Employees at Cohere enjoy these Perks: 🤝 An open and inclusive culture and work environment 🧑💻 Work closely with a team on the cutting edge of AI research 🍽 Weekly lunch stipend, in-office lunches & snacks 🦷 Full health and dental benefits, including a separate budget to take care of your mental health 🐣 100% Parental Leave top-up for up to 6 months 🎨 Personal enrichment
- benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement 🏙 Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend ✈️ 6 weeks of vacation (30 working days!)
Contact
- Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form https://docs.google.com/forms/d/12a6IrLdF3kI2nonKSr4tiFuz18rLQbaeYV-JM9L4o9Q/edit, and we will work together to meet your needs.
Additional details
- Who are we? Our mission is to scale intelligence to serve humanity.
- Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers.
- We like to work hard and move fast to do what’s best for our customers.
- Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft.
- Each person is one of the best in the world at what they do.
- We believe that a diverse range of perspectives is a requirement for building great products.
- Join us on our mission and shape the future! Why this role? We are on a mission to build machines that understand the world and make them safely accessible to all.
- Machines (or Large Language Models to be exact) learn in similar ways to humans: by way of feedback.
- By labeling, ranking, auditing, prompting, red teaming, and correcting output, you will improve Large Language Model’s performance for iterations to come, thus having a lasting impact on Cohere’s tech.
- This role is best suited for candidates who bring strong contextual judgment, cultural and bias sensitivity, and