Note: The job is a remote job and is open to candidates in USA. Mitratech is a company focused on building innovative products for the Legal, Risk, Compliance, and HR sectors. They are seeking a Senior Engineering Manager for their AI Quality & Governance function, responsible for leading the AI evaluations and observability platform while establishing standards for production-ready AI systems.
Responsibilities
- Own the AI evaluations and observability platform — tracing, logs, dashboards, quality signals, and operational feedback loops for prompts, model calls, tool use, and user outcomes
- Design and operationalize automated and human-in-the-loop evaluation strategies, integrating regression checks for quality, safety, latency, and cost into engineering and release processes
- Establish the standards that define production-ready AI across the organization — evaluation criteria, release gates, incident playbooks, and long-term quality metrics
- Build and operate AI guardrails and policy enforcement capabilities: content controls, PII detection and redaction, audit logging, and request- or workflow-level policy checks
- Translate emerging governance and risk expectations into working engineering systems and platform controls rather than static documentation
- Own platform-level SLOs and tier-two incident support for AI behavior issues, partnering with product teams who remain first-line owners for features they ship
- Act as an internal authority on AI quality and governance and participate in customer-facing conversations where product quality, safety, observability, or governance posture must be explained credibly
- Hire, mentor, and grow the team over time — the combined function may later evolve into separate Quality/Observability and Governance/Policy teams
Skills
- 8+ years of software engineering experience including meaningful experience leading engineers as a manager or technical lead
- Strong experience designing and operating production backend systems and APIs
- Demonstrated hands-on experience building, shipping, or operating production AI systems — ideally including LLM-powered or agentic workflows
- Experience with AI observability, evaluation, or debugging systems — whether through platforms such as LangSmith or LangFuse, or through internally built equivalents
- Practical experience designing or operating AI guardrails: content filtering, redaction, access controls, or other controls around model or agent behavior
- Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001, and the ability to engage thoughtfully with legal, compliance, and security stakeholders
- A strong bias toward hands-on execution, platform thinking, and creating paved roads for other teams
- Experience in legal technology, regulated SaaS, or other environments where auditability and defensibility matter
- Experience with privacy-sensitive systems and PII handling
- Experience participating in AI incident reviews, red-team exercises, or internal review boards for production AI systems
Benefits
- Health, Dental & Vision Insurance *
- 401 (k) + Employer Match *
- Unlimited PTO + 11 Paid Holidays + 4 Annual Paid Global Wellness Days Off
- STD, LTD & Group Life Insurance
- Paid Parental Leave
- Pet Insurance
- FSA & HSA Options
- Employee Assistance Program
- Remote Work
- Career Advancement & Professional Development Opportunities
- Employee Recognition
- LinkedIn Learning Platform
Company Overview