Inference.net is seeking a Machine Learning Engineer to join our team, focusing on optimizing the performance of our cutting-edge AI inference systems. This role involves working with state-of-the-art large language models and ensuring they run efficiently and effectively at scale. You will be responsible for deploying state-of-the-art models at scale and performing optimizations to increase throughput and enable new features. This position offers the chance to collaborate closely with our engineering team and make significant contributions to open source projects, like SGLang and vLLM.
We are building a distributed LLM inference network that combines idle GPU capacity from around the world into a single cohesive plane of compute that can be used for running large-language models like DeepSeek and Llama 4. At any given moment, we have over 5,000 GPUs and hundreds of terabytes of VRAM connected to the network.
We are a small, well-funded team working on difficult, high-impact problems at the intersection of AI and distributed systems. We primarily work in-person from our office in downtown San Francisco. Our investors include A16z CSX and Multicoin. We are high-agency, adaptable, and collaborative. We value creativity alongside technical prowess and humility. We work hard, and deeply enjoy the work that we do.
Design and implement optimization techniques to increase model throughput and reduce latency across our suite of models
Deploy and maintain large language models at scale in production environments
Deploy new models as they are released by frontier labs
Implement techniques like quantization, speculative decoding, and KV cache reuse
Contribute regularly to open source projects such as SGLang and vLLM
Deep dive into underlying codebases of TensorRT, PyTorch, TensorRT-LLM, vLLM, SGLang, CUDA, and other libraries to debug ML performance issues
Collaborate with the engineering team to bring new features and capabilities to our inference platform
Develop robust and scalable infrastructure for AI model serving
Create and maintain technical documentation for inference systems
3+ years of experience writing high-performance, production-quality code
Strong proficiency with Python and deep learning frameworks, particularly PyTorch
Demonstrated experience with LLM inference optimization techniques
Hands-on experience with SGLang and vLLM, with contributions to these projects strongly preferred
Familiarity with Docker and Kubernetes for containerized deployments
Experience with CUDA programming and GPU optimization
Strong understanding of distributed systems and scalability challenges
Proven track record of optimizing AI models for production environments
Familiarity with TensorRT and TensorRT-LLM
Knowledge of vision models and multimodal AI systems
Experience implementing techniques like quantization and speculative decoding
Contributions to open source machine learning projects
Experience with large-scale distributed computing
We offer competitive compensation, equity in a high-growth startup, and comprehensive benefits. The base salary range for this role is $180,000 - $250,000, plus competitive equity and benefits including:
Full healthcare coverage
Quarterly offsites
Flexible PTO
Inference.net is an equal opportunity employer. We welcome applicants from all backgrounds and don't discriminate based on race, color, religion, gender, sexual orientation, national origin, genetics, disability, age, or veteran status.
If you're passionate about building the next generation of high-performance systems that push the boundaries of what's possible with large language models, we want to hear from you!
#J-18808-Ljbffr...Job Description Job Description Job Summary As a Data Entry Clerkat Aculabs, you will be responsible for specimen receipt and... ...entering data and working in a laboratory/ office setting. No prior experience or degree is required! If youre interested in joining our...
...Waiter/Waitress Position We have an opening for full time and part time WAITER/WAITRESS positions. Location: Museum of Fine Arts, Boston - 465 Huntington Avenue, Boston, MA 02115. Schedule: Full time and part time schedules. Wednesday through Monday, 8:00...
...Job Description & Requirements Anesthesiology Physician StartDate: ASAP Pay Rate: $416000.00 - $440000.00 Step into a pivotal role at a reputable institution and elevate patient care at a Level 3 Trauma Center. The University of New Mexico (UNM) seeks a Clinical Faculty...
...Job Description We are Looking for both New and Experienced Travel Agents Full training is provided. Join Our Team Become an... ...home, on-going training and support Work Location: Hybrid remote (in-store and at-home) Skills Excellent customer service...
...Entry Level Marketing Associate Location: Orlando, FL Schedule: Monday to Friday, 8-hour shifts Salary: $29 - $32 per hour About... ...with marketing software. Basic understanding of digital marketing concepts and social media platforms. Ability to...