data
Posted Nov 5, 2025Software Engineer, LLM Infrastructure
at fireworks
On-site
Requirements
- ABOUT US: At Fireworks, we’re building the future of generative AI infrastructure.
- We’ve been independently benchmarked as the leader in LLM inference speed and are driving cutting-edge innovation through projects like our own function calling and multimodal models.
- We’re an ambitious, collaborative team of builders, founded by veterans of Meta PyTorch and Google Vertex AI.
- In the last few months alone we launched Fireworks Training, partnered with Microsoft Azure Foundry, and published research straight from our production systems.
- (blog https://fireworks.ai/blog/frontier-rl-is-cheaper-than-you-think) - Open source agents with frontier advisors: matching frontier performance through training and harness engineering.
- (blog https://fireworks.ai/blog/open-source-agents-frontier-advisors) - The fine-tuning bottleneck is not the algorithm: integration friction and iteration speed are what actually stall teams; we documented the patterns across dozens of customer engagements.
- (blog) https://fireworks.ai/blog/fine-tuning-bottlenecks THE ROLE: As a Software Engineer on our AI Infrastructure team, you will help design the core systems that power Fireworks AI’s generative AI platform.
- You will help build infrastructure and tools that ensure the reliability, performance, quality, and availability of our AI system.
- Our mission is to make Fireworks AI the most reliable and user friendly generative AI platform in the world.
- RESPONSIBILITIES: - Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines - Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems - Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers - Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry -
- QUALIFICATIONS: - Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience) - 3 years of
- experience in software engineering, with a focus on infrastructure or machine learning systems - Strong programming skills in Python, Go, or a similar language - Proven
- experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.). - Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc) PREFERRED
- experience in software engineering, with a focus on infrastructure or machine learning systems -
- Experience with open source inference engine like vLLM, Sglang, or TRT-LLM - Contributions to open-source infrastructure or ML projects -
- Experience in building large scale ML/MLOps infrastructure WHY FIREWORKS AI?
- - Solve Hard Problems: Tackle challenges at the forefront of AI infrastructure, from low-latency inference to scalable model serving.
- - Ownership & Impact: Join a fast-growing, passionate team where your work directly shapes the future of AI—no bureaucracy, just results.
- - Learn from the Best: Collaborate with world-class engineers and AI researchers who thrive on curiosity and innovation.
- Fireworks AI is an equal-opportunity employer.
Experience
- QUALIFICATIONS: - 5+ years of
Additional details
- Our platform delivers the highest-quality models with the fastest and most scalable inference in the industry.
- A few examples of what that looks like in practice: - Frontier RL is cheaper than the mega-cluster narrative suggests: we ran cross-region rollouts using 98% sparse weight deltas and published what we learned.
- You will partner closely with our cloud infrastructure team, product team, and performance team to deliver infrastructure that bridges the gap between our customer and the ultra-performant proprietary Fireworks inference engine. KEY
- We celebrate diversity and are committed to creating an inclusive environment for all innovators.