Member of Technical Staff, Pre-Training Data

Responsibilities

• - Develop robust data modeling techniques to ensure datasets are structured and formatted for optimal training efficiency.
• - Research and implement innovative data curation methods, leveraging Cohere’s infrastructure to drive advancements in natural language processing.
• - Collaborate with cross-functional teams, including researchers and engineers, to ensure data pipelines meet the demands of cutting-edge language models.

Requirements

• We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents.
• We believe that our work is instrumental to the widespread adoption of AI.
• Join us on our mission and shape the future! Why this role? As a Machine Learning Engineer specializing in pretraining data, you will play a pivotal role in developing the data pipeline that underpins Cohere’s advanced language models.
• By combining research and engineering, you will bridge the gap between raw data and cutting-edge AI models, directly contributing to improvements in critical training metrics like throughput and accelerator utilization.
• Your work will be essential to Cohere’s mission of delivering efficient and reliable language understanding and generation capabilities, driving innovation in natural language processing.
• If you are passionate about transforming data into the foundation of AI systems, this role offers a unique opportunity to make a meaningful impact.
• You may be a good fit if you have: - Strong software engineering skills, with proficiency in Python and
• experience building data pipelines. - Familiarity with curriculum learning, data mixing and data attribution. - Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools. -
• Experience working with large-scale datasets, including web data, code data, and multilingual corpora. - Knowledge of data quality assessment techniques and experimentation with data mixtures. - A passion for bridging research and engineering to solve complex data-related challenges in AI model training.

Benefits