data
Posted 4 hours agoSenior Data Engineer
at Prolific
United KingdomRemote
Responsibilities
- Build and maintain data pipelines that power analytics, ML workloads, and product-facing applications - from internal databases, SaaS sources, and streaming systems through to the teams and services that consume them.
- Evolve our data platform alongside cloud platform engineers, using infrastructure-as-code (Terraform) and Kubernetes-based deployments (Argo) to keep the platform scalable, reliable, and self-serve.
- System Architecture: Design and implement scalable data infrastructure that accommodates our growing data volume and complexity.
- Develop data services and APIs that expose trusted data to product applications, bridging analytical and operational systems.
- Own data quality and observability, putting monitoring, testing, and alerting in place so issues are caught early and trust in the data stays high.
Requirements
- Prolific is not just another player in the AI space – we are the architects of the human data infrastructure that's reshaping the landscape of AI development.
- In a world where foundational AI technologies are increasingly commoditized, it's the quality and diversity of human-generated data that truly differentiates products and models. The role
- Technical Expertise: 3+ years building and shipping production grade data systems, with strong SQL skills and proficiency in Python or another object-oriented language (Java, Scala etc.)
- Familiarity with cloud-native infrastructure - ideally some exposure to Terraform, Kubernetes, or similar IaC and container orchestration tools.
- Experience designing data APIs or services that expose data to applications, or a strong interest in working across the analytical/operational boundary.
- Strong collaboration skills and comfort working across teams with different priorities — engineers, data scientists, AI engineers, product.
- experience with data pipeline tools (Airflow, dbt) and strong ability to optimise for performance and reliability.
- Collaborative Mindset: Ability to work across teams to understand and address diverse data needs while maintaining data integrity.
- Partner with AI engineers, data scientists, analysts, and product teams to understand their data needs and design the right solutions – not just the quickest ones.
- This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.
- We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development.
- By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breath and the best of humanity.
- Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research.
Benefits
- Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture.
Additional details
- We don't expect deep expertise in all of these, but comfort working in this kind of environment matters. •
- A pragmatic, curious mindset - you enjoy keeping up with the field and know when to reach for a new tool versus when to stick with what works.
- Quality Focus: Commitment to continuously improving product quality, security, and performance through rigorous testing and code reviews.
- Documentation: Meticulous approach to creating and maintaining architecture and systems documentation.
- Growth Orientation: Desire to continually keep up with advancements in data engineering practices and technologies.
- Problem-Solving: Exceptional analytical skills to troubleshoot complex data issues and implement effective solutions.
- Independence: Capability to ship medium features independently while contributing to the team's overall objectives.
- Technical Documentation: Create and maintain comprehensive documentation of data flows, models, and systems for knowledge sharing.
- Security and Compliance: Ensure all data systems adhere to security best practices and compliance requirements.
- We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback.