engineering
Posted Feb 6, 2025Software Engineer, Model Inference
at openai
San Francisco, United StatesOn-site
Responsibilities
- - Build tools to give us visibility into our bottlenecks and sources of instability and then design and implement solutions to address the highest priority issues.
Requirements
- We empower consumers, enterprise and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they’ve never been able to before.
- About the Role We are looking for an engineer who wants to take the world's largest and most capable AI models and optimize them for use in a high-volume, low-latency, and high-availability production and research environment. In this role,
- you will: - Work alongside machine learning researchers, engineers, and product managers to bring our latest technologies into production.
- - Optimize our code and fleet of Azure VMs to utilize every FLOP and every GB of GPU RAM of our hardware.
- You might thrive in this role if you: - Have an understanding of modern ML architectures and an intuition for how to optimize their performance, particularly for inference. - Own problems end-to-end, and are willing to pick up whatever knowledge you're missing to get the job done. - Have at least 5 years of professional software engineering experience. - Have or can quickly gain familiarity with PyTorch, NVidia GPUs and the software stacks that optimize them (e.g.
- NCCL, CUDA), as well as HPC technologies such as InfiniBand, MPI, NVLink, etc. - Have
- About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence
Benefits
- Bonus point if worked on performance-critical distributed systems. - Have needed to rebuild or substantially refactor production systems several times over due to rapidly increasing scale. - Are self-directed and enjoy figuring out the most important problem to work on. - Have a humble attitude, an eagerness to help your colleagues, and a desire to do whatever it takes to make the team succeed.
Additional details
- We focus on performant and efficient model inference, as well as accelerating research progression via model inference.
- - Work alongside researchers to enable advanced research through awesome engineering.
- - Introduce new techniques, tools, and architecture that improve the performance, latency, throughput, and efficiency of our model inference stack.
- experience architecting, building, observing, and debugging production distributed systems.