attention head AI jobs Alerts Saved
AI jobs › Inference & serving AI jobs

Inference & serving AI jobs

Once a model is trained, every request to it is inference, and for a widely used model that adds up to a great deal of compute. Inference and serving teams make models answer quickly, handle many requests at once and cost less per token without making the answers worse, and, for models on phones and other devices, make them small enough to run there.

The work includes batching requests together, managing the memory the attention cache takes, quantization (storing weights and activations in fewer bits), speculative decoding (a small model drafts tokens that the large one checks), and splitting a large model across several GPUs. Inference engineers and ML systems engineers work on engines such as vLLM, which started at UC Berkeley, and NVIDIA's TensorRT; infrastructure engineers run the fleets, routing and autoscaling around them. Expect C++ or Rust beside Python, profiling, and a working knowledge of GPUs.

We count roles whose postings mention inference, model serving, vLLM, TensorRT, quantization or speculative decoding. Inference on its own also means statistical or causal inference, so some data science and research roles are counted here too.

1,911 open AI roles at 766 employers, from their own job boards. Showing the newest 100.

Follow Inference & serving117 new this week

New Inference & serving roles by email, the morning after we find them, in one email with your other alerts. Narrow it, or keep every role.

How often
We email a link to confirm it.
NVIDIA · 137Amazon · 94Cerebras · 32Anthropic · 21Fireworks AI · 20Red Hat K.K. · 19Together AI · 19OpenAI · 18Nebius · 16Waymo · 16Inferact · 15CoreWeave · 14DigitalOcean · 13Lila Sciences · 12Baseten · 11

Get new Inference & serving roles every Monday

A job alert: new Inference & serving roles from employers' own job boards, every Monday. Unsubscribe in one click.

Related