Machine Learning Engineer - Model Performance Job at Inference, San Francisco, CA

WUFkUDdJVTRqcjJiSnJNbVRvOVJiL1NVcUE9PQ==
  • Inference
  • San Francisco, CA

Job Description

Inference.net is seeking a Machine Learning Engineer to join our team, focusing on optimizing the performance of our cutting-edge AI inference systems. This role involves working with state-of-the-art large language models and ensuring they run efficiently and effectively at scale. You will be responsible for deploying state-of-the-art models at scale and performing optimizations to increase throughput and enable new features. This position offers the chance to collaborate closely with our engineering team and make significant contributions to open source projects, like SGLang and vLLM.

About Inference.net

We are building a distributed LLM inference network that combines idle GPU capacity from around the world into a single cohesive plane of compute that can be used for running large-language models like DeepSeek and Llama 4. At any given moment, we have over 5,000 GPUs and hundreds of terabytes of VRAM connected to the network.

We are a small, well-funded team working on difficult, high-impact problems at the intersection of AI and distributed systems. We primarily work in-person from our office in downtown San Francisco. Our investors include A16z CSX and Multicoin. We are high-agency, adaptable, and collaborative. We value creativity alongside technical prowess and humility. We work hard, and deeply enjoy the work that we do.

Responsibilities

  • Design and implement optimization techniques to increase model throughput and reduce latency across our suite of models

  • Deploy and maintain large language models at scale in production environments

  • Deploy new models as they are released by frontier labs

  • Implement techniques like quantization, speculative decoding, and KV cache reuse

  • Contribute regularly to open source projects such as SGLang and vLLM

  • Deep dive into underlying codebases of TensorRT, PyTorch, TensorRT-LLM, vLLM, SGLang, CUDA, and other libraries to debug ML performance issues

  • Collaborate with the engineering team to bring new features and capabilities to our inference platform

  • Develop robust and scalable infrastructure for AI model serving

  • Create and maintain technical documentation for inference systems

Requirements

  • 3+ years of experience writing high-performance, production-quality code

  • Strong proficiency with Python and deep learning frameworks, particularly PyTorch

  • Demonstrated experience with LLM inference optimization techniques

  • Hands-on experience with SGLang and vLLM, with contributions to these projects strongly preferred

  • Familiarity with Docker and Kubernetes for containerized deployments

  • Experience with CUDA programming and GPU optimization

  • Strong understanding of distributed systems and scalability challenges

  • Proven track record of optimizing AI models for production environments

Nice to Have

  • Familiarity with TensorRT and TensorRT-LLM

  • Knowledge of vision models and multimodal AI systems

  • Experience implementing techniques like quantization and speculative decoding

  • Contributions to open source machine learning projects

  • Experience with large-scale distributed computing

Compensation

We offer competitive compensation, equity in a high-growth startup, and comprehensive benefits. The base salary range for this role is $180,000 - $250,000, plus competitive equity and benefits including:

  • Full healthcare coverage

  • Quarterly offsites

  • Flexible PTO

Equal Opportunity

Inference.net is an equal opportunity employer. We welcome applicants from all backgrounds and don't discriminate based on race, color, religion, gender, sexual orientation, national origin, genetics, disability, age, or veteran status.

If you're passionate about building the next generation of high-performance systems that push the boundaries of what's possible with large language models, we want to hear from you!

#J-18808-Ljbffr

Job Tags

Work at office, Flexible hours,

Similar Jobs

HCA Houston Healthcare

Orthopedic Spine Surgeon (Houston) Job at HCA Houston Healthcare

 ...Join to apply for the Orthopedic Spine Surgeon role at HCA Houston Healthcare Continue with Google Continue with Google 2 weeks ago Be among the first 25 applicants Join to apply for the Orthopedic Spine Surgeon role at HCA Houston Healthcare Job Summary... 

The Structures Company

Electrical Install Assembler - Free Housing Support Job at The Structures Company

 ...JOB TITLE: Electrical Install Assembler - Free Housing Support LOCATION: North Charleston, SC PAY RATE: $25-31/hour We are...  ...with: Reading and interpreting blueprints and technical work instructions (in English) Using hand, power, and pneumatic... 

P2P

Senior / Game Designer (San Francisco) Job at P2P

 ...We are looking for a creative and driven Senior / Game Designer to join our team! In this role, youll contribute to the design of our...  ...Curious about our game? Take a sneak peek by checking out this video and learn all about our vision here ! Our Hiring Process... 

Harbour Dental Care

Associate Dentist Job at Harbour Dental Care

 ...Harbour Dental Care is a PRIVATE practice that EMPOWERS our Dentists by removing barriers to provide PREMIERE EXPERIENCES within their...  ...company leaders that support each location. Dr. Snyder and associate doctors are available to collaborate with. We will throw as... 

Medium

Central Editor Job at Medium

This position must sit in CST time zone. Office or remote options.Bisnow Overview Bisnow is the commercial real estate industry's leading...  ...hyper-locally.Bisnow is seeking a seasoned, razor-sharp Editor to help lead, report and build our burgeoning commercial real estate...