Senior LLM Serving Optimization Engineer
1 day ago
Winnipeg, Manitoba, Canada
Capgemini Engineering
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
Optimize LLM serving as a senior machine learning systems engineer, working remotely across Canada with vLLM, PyTorch, and accelerator workloads. The mandate spans production inference.
Capgemini Engineering’s engineer extends model coverage and tunes inference behavior across AWS Neuron and Trainium platforms. You will read and modify implementations involving attention, mixture-of-experts routing, and KV-cache management. Responsibilities include hybrid state-space models, kernel-level optimization, and clear communication of engineering trade-offs.
Key Responsibilities:
Extend model coverage across hybrid and mixture-of-experts architectures Reuse and generalize existing inference implementations where practical Optimize prefill, decode, batching, and parallelism behavior Profile framework, compiler, or kernel-level accelerator workloads Document implementation choices and benchmark reproducibility Requirements:
Strong Python and PyTorch skills with model implementation experience Hands-on LLM serving framework experience such as vLLM Deep understanding of LLM inference performance trade-offs Experience designing rigorous performance experiments Kernel development using NKI, Triton, or CUDA preferred Use LLM serving, model architecture, and accelerator optimization expertise to advance production inference systems at Capgemini Engineering.
Extend model coverage across hybrid and mixture-of-experts architectures Reuse and generalize existing inference implementations where practical Optimize prefill, decode, batching, and parallelism behavior Profile framework, compiler, or kernel-level accelerator workloads Document implementation choices and benchmark reproducibility Requirements:
Strong Python and PyTorch skills with model implementation experience Hands-on LLM serving framework experience such as vLLM Deep understanding of LLM inference performance trade-offs Experience designing rigorous performance experiments Kernel development using NKI, Triton, or CUDA preferred Use LLM serving, model architecture, and accelerator optimization expertise to advance production inference systems at Capgemini Engineering.