Senior LLMOps Engineer

About HiringBranch

At HiringBranch, we're redefining the talent acquisition game by designing conversational assessments that let candidates demonstrate their skills. 

We've proven that technology can assess soft skills more accurately and fairly than people can. Our mission is to help hiring teams make excellent hiring choices; ethically, effortlessly, and confidently, through skills-first automation.

You'll play a key role in advancing this mission while shaping the future of HR tech.

Your Mission

We're looking for a
Senior LLMOps

to design and manage the infrastructure powering our next-generation conversational AI platform. You'll ensure our ML systems are robust, scalable, and efficient, enabling real-time, LLM-powered simulations that help global enterprises hire better, faster, and with confidence.

About the Project

We're building an AI-powered platform that simulates real conversations using LLMs to measure communication, empathy, and decision-making skills.

As an essential member of the project pod, you will be responsible for ensuring our ML infrastructure is scalable, robust, and efficient, from experimentation to deployment.

What You'll Do

Core LLM Infrastructure & Deployment

• Design, build, and maintain
  infrastructure for LLM-powered simulations

, via direct model serving (e.g., OpenAI, Gemini, Claude, local models) or orchestration frameworks like LangChain.
- Develop and manage
deployment and integration pipelines

for both ML and backend services.
- Automate
training, evaluation, and deployment workflows

for prompts, agents, and scoring components.

Collaboration & Optimization

• Collaborate closely with team members, including ML Engineers, Backend Developers, and Product Managers, to define infrastructure needs and system architecture.
• Work with the
  Legal and Compliance teams

to ensure all LLMOps practices meet internal and external regulatory requirements, specifically for
GDPR and SOC 2

.
- Optimize cloud usage (
AWS

) for cost-efficiency and scale.
- Work with the cloud optimization team with tooling to identify overspending and paths to cost-efficiency.

.
GDPR & SOC 2 Compliance

• Implement data pseudonymization and anonymization techniques

for all data used in LLM training, evaluation, and logging, particularly for any
personally identifiable information (PII)

, to adhere to
GDPR's

data minimization principle.
- Establish and enforce strict access controls

(e.g., least privilege) across the LLM infrastructure and data stores to meet the
Security

and
Confidentiality

principles of
SOC 2

.

Monitoring, Logging, & Reporting

• Implement and manage
  monitoring, logging, and reporting systems

for model and infrastructure performance.
- Provide a robust logging and monitoring mechanism

that is
tamper-proof

and includes detailed metadata (e.g., source, time, model version) necessary for both operational debugging and
compliance audits

.

What Success Looks Like

• You share our passion for transforming hiring through AI.
• You engage in respectful, constructive collaboration and challenge ideas with data and empathy.
• You continually improve both our systems and yourself.
• You deliver a measurable impact on uptime, scalability, and cost efficiency.

Job Requirements

• 2+ years of experience in LLMOps, preferably in a SaaS or AI startup.
• 5+ years of experience in cloud engineering.
• Proven experience with cloud platforms (AWS), containerization (Docker, Kubernetes), and infrastructure-as-code tools (Terraform, Helm).
• Experience with deploying and maintaining LLMs and conversational AI frameworks (e.g. Gemini API, OpenAI API).
• Experience integrating LLMs into rule-based or orchestration pipelines.
• Familiarity with monitoring and observability tools (Prometheus, Grafana, ELK, Datadog, etc.).
• Strong problem-solving, analytical, and communication skills

Salary + benefits

We're offering a competitive salary, plus flexible 100% remote working, stock options, and a focus on nurturing your long-term career goals.

Why Join Us

• Shape a new AI-driven product from the ground up, with real-world impact in agent evaluation and conversational AI.
• Collaborate with an interdisciplinary, mission-driven team of ML engineers, linguists, and designers.
• 100% remote flexibility, stock options, and a focus on your long-term growth.
• Work in a
  diverse and inclusive

environment where merit matters more than background.