Senior AI Observability Systems Engineer

2 days ago

Remote, Canada Grafana Labs Full-time

Extend Tempo for AI and agent workloads through structured APIs, observability systems, and scalable engineering. Work remotely with modern coding assistants.

As a Senior Engineer on Tempo, you will build capabilities for Grafana’s AI assistant, external integrators, and agent-generated workloads. You will design deterministic interfaces, strengthen ingestion and query guardrails, and support trace aggregation and higher-density APIs. The role combines technical direction, production operations, open-source contribution, and mentoring across a globally distributed team.

Key Responsibilities:

  • Build APIs for humans, agents, and external integrators
  • Support agent-scale ingestion and high-cardinality query workloads
  • Extend TraceQL metrics and LLM-friendly response types
  • Contribute to open-source Tempo and review contributions
  • Participate in on-call, incidents, and post-incident learning

Requirements:

  • Experience building for AI or LLM consumers
  • Experience with tracing, OpenTelemetry, or observability platforms
  • Distributed data systems experience in production environments
  • Open-source contribution or maintainership experience
  • Experience operating multi-tenant infrastructure on Kubernetes

Use AI-aware API design, observability expertise, and production systems experience to expand Tempo for agents and Grafana users.

#J-18808-Ljbffr