Inference & serving AI jobs
Once a model is trained, every request to it is inference, and for a widely used model that adds up to a great deal of compute. Inference and serving teams make models answer quickly, handle many requests at once and cost less per token without making the answers worse, and, for models on phones and other devices, make them small enough to run there.
The work includes batching requests together, managing the memory the attention cache takes, quantization (storing weights and activations in fewer bits), speculative decoding (a small model drafts tokens that the large one checks), and splitting a large model across several GPUs. Inference engineers and ML systems engineers work on engines such as vLLM, which started at UC Berkeley, and NVIDIA's TensorRT; infrastructure engineers run the fleets, routing and autoscaling around them. Expect C++ or Rust beside Python, profiling, and a working knowledge of GPUs.
We count roles whose postings mention inference, model serving, vLLM, TensorRT, quantization or speculative decoding. Inference on its own also means statistical or causal inference, so some data science and research roles are counted here too.
1,911 open AI roles at 766 employers, from their own job boards. Showing the newest 100.
Follow Inference & serving117 new this week
- Robotics Machine Learning EngineerPantograph · San Franciscoup to $260K · 7 h ago
- MLOps / DevOps Engineer – AI InfrastructureCreatorIQ · New York; New Jersey; Los Angeles; San Franciscoup to $155K · 9 h ago
- Senior Applied AI/ML Scientist - Retailer GrowthFaire · San Francisco, CAup to $291K · 9 h ago
- Senior AI Platform EngineerServiceNow · West Palm Beach, Florida, United States10 h ago
- Engineering Leader, Machine Learning & Ad PlatformMoxxi Digital · New York, New Yorkup to $300K · 10 h ago
- Head of Machine LearningVidea · Boston, Massachusettsup to $275K · 10 h ago
- Senior Manager, Applied AI & Machine LearningVirta Health · San Francisco, CA; Remote; New York, NYup to $270K · 12 h ago
- Research Intern - AI /Machine Learning Numerics & EfficiencyMicrosoft · Mountain View, CA, US; Redmond, WA, US · 2 locationsup to $13K/mo · 13 h ago
- Member of Technical Staff, Core InfrastructureParallel Web Systems · San Francisco; New York, New York; Palo Altoup to $300K · 13 h ago
- AI Engineer - Agent Infrastructure & AI WorkflowTribalScale · Remote USA15 h ago
- Staff Applied Scientist, Location IntelligenceRelay Technologies · London - Hybrid19 h ago
- MLOps Engineer (CDI)Mirakl - Labs · Bordeaux, France · 2 locations19 h ago
- Applied AI Engineer (LLM)Sporty Group · EMEA21 h ago
- Forward Deployed AI Engineer, Singapore (Senior/Principal Level)Cloudera Inc. · Singapore > Singapore1 days ago
- Senior Machine Learning Engineer, Decision IntelligenceHumana Insurance Company · Remote Nationwideup to $147K · 1 days ago
- Senior Artificial Intelligence Machine Learning ScientistGM do Brasil Ltda · 5 Locationsup to $261K · 1 days ago
- AI Operations EngineerBrigham Young University · Provo, UTup to $136K · 1 days ago
- Sr. Machine Learning EngineerWorkday, Inc. · 2 Locations1 days ago
- Applied Scientist, Amazon Leo Satellite Build SystemsAmazon · Bellevue, Washington, USA · 2 locations1 days ago
- Distinguished, Machine Learning EngineerSam's East, Inc. · IN KA BANGALORE Home Office PTPP11 days ago
- Research Intern - AI Serving Systems and Hardware - Microsoft ResearchMicrosoft · Vancouver, BC, CAup to CA$10K/mo · 1 days ago
- Ingénieur(e) IA en déploiement avancé, Travel Intelligence// Forward Deployed AI Engineer, Travel IntelligenceHopper · Montréal - Remote; Québec - Remoteup to CA$425K · 1 days ago
- Member of Technical Staff - GPU Inference EngineerLiquid AI · San Francisco1 days ago
- Forward Deployed Engineer, ML Infrastructure & InferenceBlack Forest Labs · San Francisco (United States)up to $395K · 1 days ago
- Senior Machine Learning EngineerNabla · Paris office1 days ago
- Software Engineer - Inference ProductBaseten · San Francisco; New Yorkup to $400K · 1 days ago
- Senior Machine Learning EngineerHaystack News · Fort Lauderdale Office1 days ago
- Sr. Inference Engineer (LLM Serving)GoodData.AI · Prague1 days ago
- Member of Technical Staff - Applied AIATOMS Careers page · San Francisco, CAup to $321K · 1 days ago
- Staff ML Platform Engineer - Fintech Engineering (m/f/n)InPost · Warszawa / Kraków, Poland1 days ago
- Senior Machine Learning EngineerASOS · London, England, United Kingdom1 days ago
- Senior Machine Learning Engineer (MLOps)ASOS · London, England, United Kingdom1 days ago
- Robotics Research Engineer (VLA Engineer)Trener · San Jose (US-HQ)2 days ago
- MTS, Supercomputing NetworkingEtched · San Jose2 days ago
- AI/ML EngineerDeepIntent · Pune, India2 days ago
- Senior Software Engineer, AI Memory and WorkloadHP · Houston, TX2 days ago
- Lead Data Scientist (Causal Inference/Experimentation)Allstate Insurance Company of Canada - Permanent · US - Remote2 days ago
- Senior AI EngineerAristocrat Technologies Inc · Barcelona, Spain2 days ago
- MLOps Senior Lead EngineerNVIDIA · 2 Locations2 days ago
- Principal AI Governance ScientistCentene Management Company LLC · 15 Locationsup to $249K · 2 days ago
- Sr. Applied Scientist, Amazon Leo, Amazon Leo Networking SoftwareAmazon · Redmond, Washington, USA2 days ago
- Engineer Lead, Artificial Intelligence / Machine Learning- 8 to 12 Years- BangaloreFIS INC (OLD FIS) · IND BNGL FL2-3 TWR 32 days ago
- Senior Software Test Development Engineer - AI Infra, SDETNVIDIA · China, Shanghai2 days ago
- GTM & Sales Lead- Frontier AI & NCPs (East Coast)F5 Networks Australia Pty. Limited · 5 Locationsup to $230K · 2 days ago
- Senior ML Data Platform Engineer | $165K-$175K + Hybrid + Equity | AI Powered Outage Intelligence SaaS StartupPhillyTech.Co · King of Prussia, PA, United States2 days ago
- Intern, LLM Inference Acceleration Architecture (Distributed Memory-Centric Accelerators)Samsung Semiconductor · San Jose, California, United Statesup to $68/h · 2 days ago
- Intern, Architecture Research Engineer (Agentic AI Workload Characterization)Samsung Semiconductor · San Jose, California, United Statesup to $68/h · 2 days ago
- Member of Technical Staff - Product Engineer (Console/API)Liquid AI · San Francisco; Boston2 days ago
- AI Applied Research PhD Intern, Summer 2027Crusoe · San Francisco, CA - US; Sunnyvale, CA - US2 days ago
- Research Engineer, Agentic SystemsRamp · Remote (US)up to $352K · 2 days ago
- Applied AI Engineering LeadPeregrine Technologies · San Francisco, CAup to $320K · 2 days ago
- Lead DevOps EngineerSTR · Arlington, VA · 4 locationsup to $225K · 2 days ago
- Senior Machine Learning Engineer - SearchNextdoor · Dallas, TX · 2 locationsup to $276K · 2 days ago
- Research Internship – AI for 5G O-RAN: GPU & DSP Algorithms for the Physical Layer - Paris - H/FIliad - Free · Paris, IDF, France2 days ago
- Senior Machine Learning Scientist, Applied Causal InferenceDoorDash USA · San Francisco, CA; Sunnyvale, CA; Seattle, WAup to $202K · 2 days ago
- Machine Learning Engineer, RoboticsPath Robotics · Columbus, Ohio or Remote2 days ago
- AI Platform ArchitectGraphcore · Austin, Texas, United States2 days ago
- Backend Software Engineer, ML InferencePrior Labs · Berlin; Freiburg; New Yorkup to €210K · 2 days ago
- Staff AI Platform Engineer, Infrastructure Services ( Bangalore/Noida)SentinelOne · Bengaluru, Karnataka, India; Noida2 days ago
- Software Engineer, Machine LearningEma · Mountain View (Hybrid) · 2 locations3 days ago
- Research Engineer – Neuro-Symbolic AI (Knowledge Graphs) & Multimodal Assistant SystemsBosch Group · bengaluru, India · 2 locations3 days ago
- Staff Technical Program Manager, Physical AIApptronik · Austin, Texas · 2 locationsup to $280K · 3 days ago
- Staff Applied AI Engineer, Agentic SystemsBeacon AI · San Carlos - Hybridup to $260K · 3 days ago
- VP, Agentic Partnerships & GTMCircle Internet Management Services LLC · 25 Locationsup to $330K · 3 days ago
- Staff AI/ML Engineer - MTS1eBay Inc. · Austinup to $178K · 3 days ago
- Lead ML Systems Engineer - Productionization & Inference ArchitectureDigital Turbine Media, Inc. · Poland - Warsaw3 days ago
- Senior ML Systems Engineer - Productionization & InferenceDigital Turbine Media, Inc. · Poland - Warsaw3 days ago
- Senior Lead Machine Learning Engineer , AI Defense (Hybrid)Cisco Systems, Inc. · Milpitas, California, USup to $333K · 3 days ago
- Applied Scientist, Simulation Methods and Tools, Amazon Center for Quantum ComputingAmazon · Pasadena, California, USA3 days ago
- Senior Solutions Architect, HPC and AINVIDIA · 2 Locations3 days ago
- Principal Robotics Learning EngineerAIM · Seattleup to $220K · 3 days ago
- Senior DevOps Engineer (AI-Enabled Platform Engineering)EverOps · Remote3 days ago
- AI Researcher - New Grad (BS/MS)ATOMS Careers page · San Francisco, CA3 days ago
- Senior and/or Principal Software Engineer - AI InfrastructureMicrosoft · Redmond, WA, US; Mountain View, CA, USup to $235K · 3 days ago
- Cloud Platform - ML Robotics Infra EngineerATOMS Careers page · Pittsburgh, PAup to $170K · 3 days ago
- Senior ML Data Processing DeveloperLawZero · Berlin; Montreal3 days ago
- Research Engineer, Large Language ModelsImmiland Canada Inc. · Across Canada3 days ago
- ML EngineerEVB · Remote, OTHERup to $120/h · 3 days ago
- Machine Learning Infrastructure Engineer, TechnologyPoint72 · New York, NYup to $300K · 3 days ago
- Staff Engineer, Machine Learning & Data InfrastructureOKX · APAC; Singapore, Singapore · 2 locationsup to $322K · 3 days ago
- Member of Technical Staff, SearchCohere · United States; San Francisco; Europe; Canada; London; New York;…up to CA$535K · 3 days ago
- Inference Artist (Matte Painting/Compositing)Magnopus · St Albans, United Kingdom4 days ago
- Senior ML EngineerPoland and Middle-Eastern Europe · Bulgaria; Poland; Romania · 2 locations4 days ago
- AI Security EngineerDigiCert · India, Bangalore4 days ago
- Senior Applied ScientistUiPath · Bellevueup to $250K · 4 days ago
- Senior AI Research Engineer, Agentic Systems IntelligenceNVIDIA · 6 Locations4 days ago
- Senior Applied Scientist, Ad Measurements ScienceAmazon · New York, New York, USA · 3 locations4 days ago
- AI Research Engineer: Vision AI / VLM / Physical AICentific Global Solutions, Inc. · Remote Work( USA)4 days ago
- Senior Applied Scientist, Prime Video - Ads ScienceAmazon · New York, New York, USA4 days ago
- GenAI Engineer | LLMs, NLP & Cloud (MLOps)Synechron Technologies Pvt. Ltd._INDIA Company · 3 Locations4 days ago
- Generative AI EngineerSynechron Technologies Pvt. Ltd._INDIA Company · Bengaluru - GTP4 days ago
- Machine Learning Engineer IIIAdvance Magazine Publishers Inc. · DLF Downtown1, Chennai, IN4 days ago
- Sr. Machine Learning Engineer, AnonymizationMercor · New York City · 2 locationsup to $500K · 4 days ago
- ML Infra Engineer, ComputePhysical Intelligence · San Francisco4 days ago
- Agentic AI EngineerDefense Unicorns · Springfield, VAup to $201K · 4 days ago
- Staff Machine Learning Engineer, GrowthMetropolis · Seattle, Washington, United Statesup to $260K · 4 days ago
- Member of Technical Staff - Pre-Training Evals Infra (Inference)Reflection AI · Global Office4 days ago
- Engineering Manager – AIRadiant · United Kingdom - Remote4 days ago
- Senior AI EngineerVersion 1 · Bengaluru, KA, India4 days ago
- Member of Technical Staff – Infra, k8s, DynamoLyceum · Zürich5 days ago