research
Posted 2 hours agoLead Applied Scientist
at Prolific
London, United KingdomOn-site
Responsibilities
- Prototype AI/ML methods to improve human data quality, judgement aggregation, and AI evaluation workflows.
- Design experiments, benchmarks, and reliability tests to measure whether new methods improve quality, efficiency, or customer outcomes.
Requirements
- Prolific is building the human data infrastructure that powers the next generation of AI systems.
- As frontier AI labs scale their use of human-generated data for training, evaluation, and alignment, the way we measure quality, performance, and operational efficiency becomes increasingly important. The Role
- As an Applied Scientist, you will design and prototype AI/ML methods that improve data quality, scale human judgement, and support robust AI evaluation workflows.
- You will work on applied problems such as quality modelling, judgement aggregation, evaluation design, LLM-assisted review, and reliability testing for AI systems.
- Ideal for someone with deep scientific judgement, strong applied ML skills, and a practical bias toward methods that work in real customer and product contexts.
- This is not a pure research role or a production ML engineering role.
- Apply classical ML, statistics, LLMs, and agentic techniques where they create practical value.
- Use modern AI tools to accelerate prototyping, experimentation, and iteration.
- PhD or MSc in Computer Science, Mathematics, Statistics, Machine Learning, or a related field.
- experience with demonstrated real-world impact. •
- Experience with human-in-the-loop AI systems, including RLHF, annotation pipelines, data quality modelling, judgement aggregation, benchmarks, or AI evaluation.
- Fluency with modern LLM and agentic techniques, such as Retrieval-Augmented Generation (RAG), LLM-as-judge, multi-agent workflows, synthetic data generation, and automated quality review.
- Strong Python skills and the ability to quickly build, test, and iterate on working prototypes.
- Good judgement on when to use simple statistical methods, classical ML, LLMs, or agentic approaches.
- experience partnering with product and engineering teams.
- This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.