Engineering Testing Manager
16 hours ago
Toronto ON, Toronto Census Division, ON; Ontario, Canada
Cerebras
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
This architecture allows Cerebras to deliver industry-leading training and inference speeds;
This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.
We route traffic across all of Cerebras’ data centers and third-party integrations, balancing heterogeneous compute, maximizing cache utilization, and delivering the industry’s lowest time-to-first-token (TTFT) at massive scale.
As an Engineering Manager, Inference Router , you will lead the team responsible for the intelligent control plane of the fastest generative AI inference platform in the world. It combines technical leadership, people management, and execution ownership, with direct impact on the latency, throughput, and reliability experienced by every Cerebras inference customer.
Own the architecture and evolution of the inference routing layer across all Cerebras data centers and third-party inference integrations.
routing for heterogeneous compute, including prefill/decode (P/D) disaggregated serving;
Invest in simulation: build high‑fidelity models of the fleet and its workloads so the team can validate routing policies, failover behavior, and scaling decisions under a wide range of conditions without tying up precious accelerator capacity.
Partner with inference runtime, cloud/platform, hardware, and ML teams to optimize end‑to‑end performance.
Provide technical direction, mentorship, and career development.
prefix‑ and cache‑aware routing policies;
autoscaling and capacity management across heterogeneous fleets;
simulation and load‑generation frameworks that exercise the router under diverse traffic patterns, failure modes, and fleet topologies — without consuming scarce production compute;
Ensure high‑quality releases through strong testing, load/chaos validation, and operational rigor.
Platform & Performance Ownership
Drive improvements in TTFT, throughput, cache‑hit rate, and compute efficiency across the fleet.
Identify and prioritize technical debt and system bottlenecks.
8+ years of experience in large‑scale software engineering, distributed systems, or infrastructure.
~2+ years of engineering management experience.
~ Strong programming skills in C++ and/or Go for performance‑critical, concurrent production systems.
~ Deep experience operating production services on Kubernetes : deployments, autoscaling, rollout strategies, and multi‑cluster/multi‑region topologies.
~ Proven track record building and operating highly available, scalable systems: load balancing, failover, scale‑out, and reliability engineering.
~ Experience with cloud infrastructure and best practices for building scalable microservices.
Experience with large-scale LLM/inference serving and the systems around it (e.
ML background: understanding of inference workloads, batching, and the performance characteristics of serving LLMs and multimodal models. Experience building simulation, modeling, or load‑testing frameworks for distributed systems to validate behavior cost‑effectively at scale. The routing decisions your team makes — across data centers, across heterogeneous compute, with cache‑awareness and high availability — directly determine the latency, throughput, and reliability every customer experiences. People who are serious about software make their own hardware. Our simple, non‑corporate work culture that respects individual beliefs. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them. This website or its third‑party tools process personal data.
ML background: understanding of inference workloads, batching, and the performance characteristics of serving LLMs and multimodal models. Experience building simulation, modeling, or load‑testing frameworks for distributed systems to validate behavior cost‑effectively at scale. The routing decisions your team makes — across data centers, across heterogeneous compute, with cache‑awareness and high availability — directly determine the latency, throughput, and reliability every customer experiences. People who are serious about software make their own hardware. Our simple, non‑corporate work culture that respects individual beliefs. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them. This website or its third‑party tools process personal data.